FROBENIUS-SCHUR FUNCTIONS 



Grigori Olshanski, Amitai Regev, and Anatoly Vershik 
(with an appendix by Vladimir Ivanov) 

Abstract. We introduce and study a new basis in the algebra of symmetric func- 
tions. The elements of this basis are called the Frobenius-Schur functions (PS- 
functions, for short). 

Our main motivation for studying the PS-functions is the fact that they enter a 
formula expressing the combinatorial dimension of a skew Young diagram in terms of 
the Frobenius coordinates. This formula plays a key role in the asymptotic character 
theory of the symmetric groups. The .PS-functions are inhomogeneous, and their 
top homogeneous components coincide with the conventional Schur functions (S- 
functions, for short). The PS-functions are best described in the super realization of 
the algebra of symmetric functions. As supersymmetric functions, the PS-functions 
can be characterized as a solution to an interpolation problem. 

Our main result is a simple determinantal formula for the transition coefficients 
between the PS- and S-functions. We also establish the FS analogs for a number of 
basic facts concerning the S- functions: Jacobi-Trudi formula together with its dual 
form; combinatorial formula (expression in terms of tableaux); Giambelli formula 
and the Sergeev-Pragacz formula. 

All these results hold for a large family of bases interpolating between the PS- 
functions and the ordinary S-functions. 
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§0. Introduction 

Let A denote the algebra of symmetric functions. Recall that A is a graded 
algebra, isomorphic to the algebra of polynomials in the power sums Pi,p%, . . . . A 
natural homogeneous basis in A is formed by the Schur functions (or S 1 - functions, 
for short). An S'-function is denoted as s M , where the index \i is a Young diagram. 
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Let x = (x\, X2, ■ ■ ■ ) and y = (yi, y2> • • • ) be two infinite collections of indeter- 
minates. We will mainly deal with the super realization of A, which is defined by 
the following specialization of the generators Pi, J>2, • • • : 



Pk -> Pk (x;y) = J2 x ^ + (- 1 ) k ^Vi- 
i=i i=i 

In this realization, each element F G A turns into a supersymmetric function in 
x,y, which will be denoted as F(x;y). In particular, the supersymmetric Schur 
function will be denoted as s^(x;y). 

Any supersymmetric function F(x;y) is separately symmetric in a^'s and y/s, 
and moreover, when a substitution Xi = —yj = t is made in F, the result does 
not depend on t. The latter property should be viewed as 'invariance' under the 
nonexisting 'supertransposition' Xi <-> yj. 

Let /i and f be Young diagrams. We are interested in the combinatorial function 

dim(n, v) 

v i-> —r- » 

dim z/ 

where \i is fixed while z/ ranges over the set of Young diagrams; dim v is the number 
of standard tableaux of shape v\ dim( / u, v) is the number of standard tableaux of 
skew shape v j \x provided that fj, C i/, and otherwise. 
Write v in the Frobenius notation: 

v = (p 1 ,...,p d \q 1 ,...,q d ), 

where d = d{v) (the depth of v) is the number of diagonal squares in v, Pi = Vi — i, 
qi =v[ — i, and v' stands for the transposed diagram. Let m = n = where 
| • | denotes the number of squares of a Young diagram. Note that the function s M 
is homogeneous of degree \fx\. 

Our starting point is the following result: if m < n then 

dim(//, v) = s tl (x(u);y(u)) + ... , Q ^ 

dim v n(n — 1) . . . (n — m + 1) ' 

where (x(v);y(v)) are the so-called modified Frobenius coordinates of v: 

[x{y);y{y)) = ( Pl + ±, . . . , Pd + ±, 0, 0, . . . ; qi + \, . . . , q d + |, 0, 0, . . . ), (0.2) 

and the dots in the numerator denote the remainder term, which is a (super) symmetric 
function of degree strictly less than |//|, evaluated at (x(v);y(v)). 
An equivalent form of (0.1) is as follows ([VK], [W, III. 6]): 

Xpui"-™ = P P (x(u);y(u)) + ... 
dimz/ n(n — 1) . . . (n — m + 1) ' 

where p = (pi, p 2 , . . .) is a partition of m, x u is the irreducible character of the 
symmetric group S(n) which corresponds to v, x^p-m is the character value on the 
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conjugacy class in S(n) indexed by the partition pUl n m of n, and p p = p Pl p P2 
The equivalence of (0.1) and (0.1') follows from the relations 

Pp = Y x p s >* ' Xpui»-™ = Y x p dim (^' v ) ■ 



Formula (0.1/) was one of the basic ingredients in the asymptotic approach to 
Thoma's description of characters of the infinite symmetric group [T], see [VK], 
[KV2], [W]. Note that dimz/ is a well-known combinatorial function for which 
several nice formulas are known, so that the complexity of (0.1) is mainly caused 
by the relative dimension function dim(/j, v). 

The aim of the present paper is to study in detail the function which appears 
in the numerator of the right-hand side of (0.1). By the very definition, this is an 
inhomogeneous supersymmetric function, and by our convention, we can view it 
as an element of A. We call it the Frobenius-Schur function with index p (or the 
FS'-function, for short) and we denote it as Fs p . In this notation, (0.1) takes the 
form 

dim(/i,i/) Fs^{x{v);y(v)) 



dimz/ n(n — 1) . . . (n — m + 1) 



(0.3) 



Our main results about the FS'-functions, and their key properties are as follows: 

• The FS'-f unctions can be characterized in terms of a multivariate interpolation 
problem. 

Specifically, given a diagram p, Fs p is the only (up to a scalar multiple) super- 
symmetric function in (x;y) that vanishes at (x;y) = (x(u);y(u)) when \v\ < \p\, 
y^/j, and does not vanish when v = p. 1 

This characterization follows at once from a correspondence between the FS- 
functions and the so-called shifted Schur functions (see below), and the interpola- 
tion properties of the latter functions established in [Okl], [OOl]. 

• As Fs p differs from s p in lower terms only, the F^-functions form a basis in 
A. We find explicitly the transition coefficients between these two bases and 
{Fs^}; they are given by simple determinantal expressions. Specifically, we have 



where pi,P2, ■ ■ ■ , <Zi> <Z2> • • • denote the Frobenius coordinates of fx, and p\, p' 2 , . . . , 
q[, q' 2 , ■ ■ ■ denote the Frobenius coordinates of v. The summation in (0.4) is taken 
over diagrams v with the length of the diagonal equal to that of fi (say, d) . So, the 
determinants are of order d. Finally, the coefficients c pp are as follows: 

v = ((-'^i ^ P,< ~^ (0.5) 
I 0, p' > p, 

for any p,p' = 0,1,2,..., where ei,e2, ... stand for the elementary symmetric 
functions. This implies, in particular, that only diagrams v contained in p may 
contribute. See Theorem 2.6 and the more general Theorem 7.3. 



1 Actually, vanishing holds for all v which do not contain /x. 
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• Consider the duality map uj : A — > A that sends p k to (— l) k 1 p k (in the super 
realization, (oj(F))(x;y) = F(y;x) for F e A). Then (Proposition 2.2) 



w(Fs M ) = , 

which is similar to the well-known property cu(s M ) = s M / of the Schur functions. 

• For natural FS analogs of the complete homogeneous symmetric functions, 
the elementary symmetric functions, and the hook functions, 

Fh k = Fs( fc ), Fe k = Fs^k), Fs( k \ /) = Fs^ k+ltl i^ 

we get counterparts of the well-known generating series. See formulas (2.3), (2.4), 
and Theorem 2.3. 

• There exists a wider family of 'multiparameter Schur functions' which depend 
on a doubly infinite string (aj)jgz of complex parameters and interpolate between 
the conventional Schur functions (case = 0) and the FS'-functions (case a, = 
% — \). See §3. 

• Recall that the supersymmetric Schur functions in finitely many variables can 
be obtained via the Sergeev-Pragacz formula, which is an analog of the basic formula 
for the Schur polynomials, 



det[a^ + TC - J "l 
det \x\ 



s fl (x 1 ,...,x n )= 1 ^; n _ J1 J - (0.6) 
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In Theorem 6.1 we get a version of the Sergeev-Pragacz formula for the multipa- 
rameter Schur functions (in particular, for the FS- functions). 

• Recall the classical combinatorial formula: 

s M (xi,x 2 ,...) = 5^ II 

t (ij)ev 

summed over the semi-standard tableaux T of the shape fx, and note that it also 
has a super analog. We get a version of this formula for the FS'-functions (as well 
as for multiparameter Schur functions), see Theorem 4.6 and the Appendix. 

• The classical Jacobi-Trudi formula and its dual version, 

s M = det[/i^_ i+i ], = det[e^_ i+i ], 

also have counterparts for the F.S-functions and multiparameter Schur functions, 
see the formulas just before Theorem 2.3, formula (3.4), and Corollary 4.9. 

• Exactly as in the classical Giambelli formula, we have 

Fs fl = det[Fs {p . lq . ) ], 

where pi, . . . , pd, qi, ■ ■ ■ , qd are the Frobenius coordinates for fx, d = d(fi). Moreover, 
the same formula holds for the multiparameter Schur functions. 

We now briefly describe the history of formula (0.1) and the relationships between 
our work and [VK], [W], [KO], [001], [Mo], [LI], [L2]. 

The formula (0.1) (or rather (0.1')) first appeared in [VK, §5] in connection with 
the asymptotic approach to Thoma's classification of the characters of the infinite 
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symmetric group. However, the original proof of (0.1') was not included in [VK] 
and remained unpublished. Then a proof was given in [W, III. 6, Theorem 6] but 
this work also remained unpublished. 

A first published proof of (0.1')) was given in [KO, Theorem 5]. This paper 
used the concept of shifted symmetric functions (see §1 for the definition). In [KO], 
it was shown that there exists a natural isomorphism between the algebra A* of 
shifted symmetric functions and the algebra of supersymmetric functions (hence, 
the algebra A). Both algebras are viewed as algebras of functions on the set of 
Young diagrams. Under this isomorphism, a shifted symmetric function in the row 
coordinates (vi, v 2 , . . . ) of a Young diagram v turns into a supersymmetric function 
in the modified Frobenius coordinates (0.2). 

The algebra A* of shifted symmetric functions was studied in detail in [001]. 
In particular, it was proved that 

dim(/x, v) = ^(z/!,z/ 2 ,...) 
dim v n(n — 1) . . . (n — m + 1) ' 

where s* G A* are the so-called shifted Schur functions. By (0.3), this means 
that under the isomorphism A* — > A mentioned above, Fs^ is the image of s*. 
For s*, a number of explicit expressions is available, see [OOl]. This provides us 
with explicit formulas for dim( / u, v) expressed in terms of the row coordinates. The 
initial purpose of our work was to get similar results in terms of the Frobenius 
coordinates. 

Another source of inspiration for our work was Molev's paper [Mo]. Molev deals 
with supersymmetric polynomials with finitely many indeterminates (xi, . . . ,x m ; 
yi,...,y n ) and studies multiparameter Schur polynomials with an arbitrary se- 
quence of parameters a = (a^gz. He obtains, among other results, generating 
series for the one-row and one-column polynomials, a combinatorial formula, and 
a Sergeev-Pragacz-type formula. However, Molev's approach is not quite consis- 
tent with our purposes, because his polynomials are not stable as m, n — > oo and, 
consequently, do not define supersymmetric functions in infinitely many indetermi- 
nates. Also, his formulas are not symmetric with respect to the duality map u that 
interchanges the x's and the y's. 

We show how to make Molev's construction stable and cu-symmetric. The recipe 
is simple: it suffices to impose the restriction m = n and to introduce in the initial 
definition of the polynomials a certain 'shift' that depends on n. In this way we come 
to a generalization of the FS'-functions depending on a = (aj). In our approach, we 
start with Molev's generating series, which needs only a slight adaptation. However, 
our versions of the combinatorial formula and the Sergeev-Pragacz-type formula 
differ from Molev's versions in a more substantial way. 

As was pointed out by Lascoux [LI], [L2], [001, §15.6], the shifted Schur func- 
tions can be introduced by means of the Schubert polynomials. Possibly, the FS 
functions can also be handled using the techniques of the Schubert polynomials. 

As is well known, there exists a deep analogy between the conventional Schur 
functions (also called Schur's S'-functions) and Schur's Q-functions. The latter are 
related to projective representations of symmetric groups just in the same way 
as the former are related to ordinary representations. Schur's Q-functions span a 
subalgebra in A, whose elements can be characterized by a supersymmetry property 
of another kind, see, e.g., [Mai, §111.8], [P]. The results of the present paper have 
counteparts for Schur's Q-functions: these are due to Ivanov [II], [12]. 
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For an application of the FS functions, see [BO, §6]. 

Finally, we would like to notice a remarkable stability of the Giambelli formula: 
in various generalizations of the Schur functions, examined in [Mai], [001], [Mo], 
and the present paper, it remains intact while the Jacobi-Trudi identity requires 
suitable modification. According to a general theorem due to Macdonald [Ma2], this 
effect holds under rather wide assumptions. The Giambelli formula is substantially 
exploited in our approach. 

The paper is organized as follows. In §1 we collect definitions and results from 
[KO], [001] which are employed in our work. In §2 we introduce the FS'-functions, 
then write down the generating series and calculate the transition coefficients be- 
tween the S- and F.S-functions. In §3 we introduce a wider family of 'multipa- 
rameter Schur functions' depending on sequence of parameters a = (a^) and we 
explain their connection with the generalized factorial supersymmetric Schur poly- 
nomials studied by Molev [Mo]. In §4 we state the combinatorial formula, which 
expresses the multiparameter Schur functions in terms of tableaux. Its proof is 
given in the Appendix, written by Vladimir Ivanov. §5 is devoted to an analog of 
the Sergeev-Pragacz formula. In §6 we calculate the transition coefficients between 
multiparameter Schur functions corresponding to different sequences of parameters. 

A short exposition of the present paper is given in [ORV] . 

Acknowledgment. We would like to thank Alain Lascoux for discussions and 
helpful critical remarks. The work was supported by RFBR grant 98-01-00303 
(G. O.); ISF grant 6629 (A. R.); CDRF grant RM1-2244 and RFBR grant 99-01- 
00098 (A. V.). G. O. and A. V. are grateful to the Weizmann Institute of Science for 
hospitality. We would also like to thank the referee for a number of useful remarks. 

1. Preliminaries on supersymmetric and shifted symmetric functions 

In this section, our main references are Macdonald's book [Mai] (symmetric func- 
tions in general), [BR] and [Mai] (supersymmetric functions), and [OOl] (shifted 
symmetric functions). We take C as the base field. 

The algebra A of symmetric functions can be initially defined as the algebra of 
polynomials C[pi, j> 2 , • • •]• Then it can be realized, in different ways, as an algebra 
of functions, depending on a specialization of the generators pk- In the conventional 
realization, the generators pk are specialized to the Newton power sums, 



where x = (xi,X2, • • •) is an infinite collection of indeterminates. Then elements 
/ G A turn into symmetric functions in x. The term 'functions' makes sense if one 
assumes, e.g., that only finitely many of different from 0. 

Let A n be the algebra of symmetric polynomials in xi, . . . ,x n . 2 In the con- 
ventional realization, A is identified with hm An., the projective limit taken in the 
category of graded algebras, where the projection A n — > A n _i is defined as the 
specialization x n = 0. 



oc 




2 To emphasize the difference between the case of infinitely many indeterminates and that of 
finitely many indeterminates, we will employ the terms 'functions' and 'polynomials', respectively. 
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We will mainly deal with another realization of A, which may be called the super 
realization. In this realization, the generators Pk are viewed as super power sums, 

oo oo 

Pk >-> Pk(x; y) = J2 x i + (- 1 ) fc_1 Yl y * = Pk ^ ~ Pk(-V) » 

where y = (yi, y 2 , • • • ) is another collection of indeterminates. Note that our defini- 
tion of the super realization of A agrees with that of [BR] but slightly differs from 
that of [Mai, Ex. 1.3.23]. 

Denote by A m n the algebra of polynomials in xi, . . . , x m , y±, . . . , y n , which are 
separately symmetric in x's and y's and which satisfy the following cancellation 
property: for any % and j, the result of the specialization Xi = —yj = t does 
not depend on t. Such polynomials are called super symmetric, and the (double) 
projective limit algebra h_mA mjn is called the algebra of supersymmetric functions 
in x and y. Here the limit is again taken in the category of graded algebras, and the 
projections A m n — > A m _i n and A m n — > A m n _i are defined by the specializations 
Xm = and y n = 0, respectively. 

By a well-known theorem (see [St], [P, Theorem 2.11]), the super realization of 
A establishes an isomorphism between the algebra A and the algebra hm A m „ of 
supersymmetric functions, so that we may identify these two algebras. For this 
reason, we will not introduce a separate notation for the algebra of supersymmetric 
functions. 

Given / G A, we write f(x; y) for the corresponding supersymmetric function in 
x and y, and we write f(x±, . . . , x m ; yi, . . . , y n ) for the supersymmetric polynomial 
that is the image of f(x; y) in A m n . In §§4-6 it will be convenient to assume m = n 
and to think about A as of the projective limit algebra \im A„ , >n . 

By the cancellation property, the algebra K n>n of supersymmetric polynomials is 
invariant under any shift of variables of the form 

(xi,...,x„;yi,...,y„) i-> (xi + r, . . . , x n + r; yi - r, . . . , y n - r), r e C. 

Let T r denote the corresponding automorphism of A nj7l : 

(T r /)(xi, . . .,x„;yi,. . .,y n ) = f{x\ + r, . . . , x n + r; yi - r, . . . , y n - r). 

Applying again the cancellation property we see that the automorphisms T r : 
K n>n — > A n>n and T r : A n+ i ;n _|_i — > A n+ i in _|_i are compatible with the canonical 
projection A n+ i in+ i — > A n n . Thus, for any r G C, we get an automorphism T r of 
the algebra A = hmA n n which informally can be written as follows: 

(T r /)(xi,x 2 , . . .;yi,y 2 , • • •) = f( x i + r,x 2 + r, . . .;y 1 - r,y 2 - r, . . .) V/ G A. 

(1.1) 

Equivalently, T r can be defined by 

i=i ^ Jy ^ 

Recall that the elements /i^ and (the complete homogeneous symmetric func- 
tions and the elementary symmetric functions) can be introduced through the gen- 
erating series: 



where t is a formal indeterminate. In the super realization, 

1 + Pit -1 , ^fc TT 1 + x i f 
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k=l i=l k=l i=l 



It will be convenient for us to take t = ^ and to redefine the generating series 
for {hk} and {e^} as formal series in ^, i.e., as elements of A[[^]]: 

fc=i fc=i 
Then their super specialization takes the form 

00 , 00 1 , 

= II B M toy) = Ul±2fc. (1.2 

Obviously, 

H{u)E{-u) = 1. (1.3) 

Proposition 1.1. in terms of the generating series (1.2), £/ie automorphisms T r 
defined in (1.1) act on £/ie elements hk and e^ as follows: 

T r (H(u)) = H(u - r), T r {E{u)) = E(u + r). 

It should be noted that for any formal series in ^ (in contrast to a series in it), 
the change of a variable w -u + const makes sense. 

Proof. This follows from the equalities 

1 + {Vi — r)/u _ 1 + Vi/{u — r) 1 + (xi + r)/u _ 1 + Xi/(u + r) 

l-(xi + r)/u l-Xi/{u-rY l-(^-r)/tt 1 - yi/(u + r) ' 

□ 

Recall that the duality map is defined as an algebra isomorphism uj : A — > A 
such that iv(pk) = (—l) h ~ 1 Pk- In the super realization, u; reduces to interchanging 
x and y. We have oo(hk) = and uj(ek) = hk- 

The Schur function s^ indexed by a Young diagram fx can be introduced through 
the Jacobi-Trudi formula: 

s M = det[/i Mi _ i+j ] , 

where, by convention, ho = 1, h_\ = h-2 = • • • = 0, and the order of the determi- 
nant is any number greater or equal to £(fi), the number of nonzero row lengths of 
\i. The Schur functions form a homogeneous basis in A (it is convenient to agree 
that sqs = 1). 

Given m and n, the supersymmetric Schur polynomial s^xi, . . . , x m ; y±, . . . , y n ) 
inm + n indeterminates does not vanish identically if and only if the diagram li 
does not contain the square (m + 1, n + 1). Such polynomials form a basis in A m>n . 

We have 



so that (dual version of Jacobi-Trudi, or Nagelsbach-Kostka formula) 



s M = det[e^._ i+j ] . 

For p, q = 0, 1, . . . , let (p \ q) denote the 'hook' Young diagram (p + 1, l 9 ), and let 
s (p\q) denote the corresponding 'hook Schur function'. 3 The hook Schur functions 
form a convenient system of generators of A. However, they are not algebraically 
independent. The relations between them are as follows: 

s (p+l\q) + s (p\q+l) = S (p\0)S(0\q) , P, Q = 0, 1, 2, . . . (1.4) 

(see, e.g., [Mai, Ex. 1.3.9]). This is equivalent to 



/ OO 7 \ / OO \ 

1 + (« + *> £ ^T> = + 1 + =Wv). (1.5) 

p,q=0 \ p=l / V q=l J 

In the Frobenius notation, a diagram is written as fx = (pi, ■ ■ ■ ,Pd\qi, • • ■ ,Qd) 
or, in more detail, 

/*= (pM,---,Pd(v) \qi(iu),...,q d (iu)), 

where <i = <i(//), the (ie^t/i of /U, is the number of diagonal squares, and Pi = Hi — i, 
Qi = K ~ i 

A very useful expression of the Schur functions is given by the Giambelli formula: 

s M = det[s (p . k .)]^ =1 . 

We proceed to the algebra A* of shifted symmetric functions. This is a filtered 
algebra such that the associated graded algebra is canonically isomorphic to A. Its 
definition is parallel to that of the algebra A in its conventional realization. First, 
let A* be the subalgebra in C[xi, . . . ,x n ] formed by the polynomials which are 
symmetric in 'shifted' variables x'j = Xj —j, j = 1, . . . , n. Define the projection map 
A* — > A* _ 1 as the specialization x n = and note that this projection preserves the 
filtration defined by ordinary degree of polynomials. Now set A* = hjnA*, where 
the limit is understood in the category of filtered algebras. 

The definition of the shifted Schur functions s* G A* is parallel to the definition 
of the conventional Schur functions via formula (0.6); the difference is that the 
ordinary powers x m are replaced by the falling factorial powers 



x lm = 



x(x — 1) . . . (x — m + 1), m > 1, 
1, m = 0, 



and a shift of variables is introduced: 



det[(xi + n-i)^i+ n -i] 

S.AXl, . . . ,X n ) = - 1 — r-; tz — . 1.6 

mV ' ' 1 det[(xi + n-i)t n -3] K 1 



3 Note that in [BR], the term 'hook Schur functions' has another meaning. 
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Clearly, this is a shifted symmetric polynomial of degree \fi\ = Yli^i- Moreover, 
the formula is stable as n — > oo and, thus, indeed determines an element of A*. We 
agree that s* = 1. 
We have 

s* (xi, . . . , x n ) = . . . , x n ) + lower terms. 

It follows that, under the canonical isomorphism gr A* = A, the highest term of s* 
coincides with s^. This implies, in particular, that the shifted Schur functions form 
a basis in A*. 

Recall the definition of the relative dimension function dim( / u, z/), where fi and 
v are arbitrary Young diagrams: 

If fj, C v then dim(//, v) is the number of standard tableaux of skew shape v j [i, 
i.e., the number of chains of diagrams of the form 



At = A C A 1 C • • • C X k 



k = \u\ — \n\ 



In particular, dim(p,,fi) = 1. Next, if \x is not contained in v then dim^u, v) = 0. 
Finally, we also set dim^ = dim(0, y), which is equal to the number of standard 
tableaux of shape fx. 

The relative dimension can be expressed in terms of the shifted Schur functions: 

Proposition 1.2. For arbitrary Young diagrams [i, v, 
dim(^) _ s*(i/i,i/ 2 ,...) 



dimu 



n 



I m 



m = \n\,n = \v\ 



Several proofs of this result are given in [001, Theorem 8.1]. The argument 
presented below is a modification of one of them. 

Outline of proof. The starting point is the well-known formula 

v. vD/J,, \v\ — \n\+l 



It follows that 



Pi 



dim v • s v 



and, more generally, 



S M Pi 



dim( / u, u) s v , = m, n > m. 



Arguing as in [Mai, Ex. 1.1.7] we get from these formulas the following ones: 



dim v = nl det 



<i<j<l 



'vi - Vj -i+j) 



dim((U, v) = (n — m)\ det 



1 



i 
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where / is any sufficiently large natural number (I > n is enough) and = T (k+i) 
equals when k = — 1, —2, .... 
This implies 



d ™ M = J- ■ det 
dimz/ n->- m 



(i/i - //j- - i + j)l 



i, j= l YllKiKjKliVi-Vj-i + j) 



After simple transformations we get 

dirndl/) _ 1 det^ + Z-z)^^^]^ 
dimz/ n-L™ rii<i<j</(^ - "j - * + j) 

as required. □ 

Proposition 1.3. For arbitrary Young diagrams fi, v, 

s* (ui, i>2, ■ ■ • ) = unless fj, C. u , (1-7) 

Proof. See [Okl] and [001, Theorem 3.1]. □ 

In the same way as for the conventional Schur functions, we set 

hk = S (k)i e t = S (lfe)5 S (p I q) = S (p+l,l9)- 

The generating series for {hi} and {e£} are defined by analogy with (1.2) but 
the ordinary powers of u are replaced by the falling factorial powers: 

00 u* 00 * 

"» = i+£^. = 

fc=i fc=i 
Proposition 1.4. T/ie specialization of these series at x = (x±, x%, . . .) tafces £/ie 
/orm 

m , w s -TT 1+V^ rn*/ x 1 + (Xi - i + 1)/U 

f=i l + f = i l + (-i + l)/u 

and the following relation holds (cf. (1.3)): 

H*{u)E*{-u- 1) = 1. (1.9) 

Proo/. See [001, §12]. □ 

Since A* is isomorphic to the algebra of polynomials in h\, h^, ■ ■ ■ , we may define 
a 1-parameter family {T r }, r e C, of automorphisms of the algebra A* by 

~~ T*JJi* k ) 
it -I 

Together with (1.9) this implies 

T r *(£») = 1 + £ = + r). 

k=i u 

It readily follows that 

T*(h%) = hi + {a linear combination of /4-i> • • • >^i> 1}; 
T*(e£) = e£ + {a linear combination of e£_ l5 . . . , e*, 1}. 

ll 



T P *(fr») = 1 + jr = ^* ( M - r )- 



Proposition 1.5. The following analogs of the J acobi-Trudi formula and its dual 
version hold: 

s ; = detir;.!^.^)] = det[Ti j+1 ( e ;,_ i+j )]. 

Proof. See [001, Theorem 13.1]. □ 

The following general result is due to Macdonald: 

Proposition 1.6. Let A be a commutative algebra and {hk, r } be a double sequence 
of elements of A. Here k = 1, 2, . . . , r = 0, 1, . . . . We also agree that /io, r = 1 and 
hk, r = for all k < 0. For an arbitrary Young diagram fx, set 

S/i = det[/i Mi _i + jj_i], 

where the order of the determinant is any number > £(fx) . Finally, set 

S (p\q) = S ( P +i,i")- 
Then the Giambelli formula holds: 

= det[S(p.\ q .)]i jj=1 , where p< = Pi{p), q t = qi{p), d = d(ji). 

Proof. See [Mai, Ex. 1.3.21] or [Ma2]. □ 

As a corollary we get the following result, which was pointed out in [001, Re- 
mark 13.2]: 

Proposition 1.7. For shifted Schur functions, the Giambelli formula remains in- 
tact: 

s* = det[s* {p . Mlq . M) }. 

Proof. We employ Proposition 1.5 and apply Proposition 1.6 to the two-parameter 
family /i fcjr = T;/i*. □ 

Following [KO], we define an algebra isomorphism ip : A* — > A by 

<p(H*(u)) = H{u+±). 

Clearly, ip intertwines the automorphisms T r : A — > A and T* : A* — > A*. The 
next result yields a characterization of (p. The key idea is to realize both A* 
and A as algebras of functions on Young diagrams; here the realization of A* is 
defined in terms of the row coordinates while the realization of A requires Frobenius 
coordinates. 

Proposition 1.8. Let /gA* be arbitrary. For any Young diagram X, 
/(Ai,A 2 ,...) =¥>(/)(pi + 5,...,Pd+5; + + 
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where \ are the row lengths of X and (pi, ■ ■ ■ ,p d \ qi, ■ ■ ■ , q d ) is Us Frobenius nota- 
tion. 

Proof. Our argument is a slight simplification of the proof given in [KO]. Let □ 
range over the squares of A and let c(n) denote the content of □, i.e., c(n) = j — i 
if □ = Using Proposition 1.4, we have 

H*(u)(\ 1 ,...,\ l ) = H U + l 



u + i — Xi 

j-r u + i u + i — 1 u + i — Xi + 1 
*-\u + i — lu + i — 2 u + i — Xi 



= n 



u - c{u) + 1 



u - c(n) 

In the latter product, we may first fix a diagonal hook and then let □ range along 
this hook. In the ith diagonal hook, the content ranges from — q t to Pi. From this 
we conclude that 

u*( \(\ w YT u ± gj ± 1 n " + Hfe + D 

ff(u)(Al ,..., A , )= n-_ = n_ I - I _ iy 

= fl"(u + |)(pi + |, . . . ,p d + \ ; q x + |, . . . , q d + \) 

(here we have used (1.2')). □ 

§2. The Frobenius-Schur functions 
We keep to the notation of §1. 

Define the Frobenius-Schur function indexed by a Young diagram fx as the fol- 
lowing element of A: 

Fs^ = <p(s;). (2.1) 

Here ip is the isomorphism A* — > A introduced just before Proposition 1.8. 
Then the following characterization theorem holds: 

Theorem 2.1. Fs^(x;y) is the only supersymmetric function such that for any 
Young diagram v = (pi, . . . ,p d \ q u . . . , q d ), 

dim(^,z/) _ Fs^( Pl + \, . . . , p d + | , ; gi + |, . . . , q d + \) 
dimv n(n — 1) . . . (n — m + 1) 

where m = \fx\, n = \v\. 

Proof. Formula (2.2) holds by virtue of Proposition 1.2 and Proposition 1.8. The 
uniqueness claim follows from the fact that a supersymmetric function is uniquely 
defined by its values on the set of the modified Frobenius coordinates of Young 
diagrams. □ 

As it was already mentioned in §0, formula (2.2) is important for the asymptotic 
proof of Thoma's theorem [T]. Generalizations of Proposition 1.2, Theorem 2.1, 
and Thoma's theorem are given in [002], [KOO], [II]. 

13 



Proposition 2.2. We have 



Proof. This follows from the fact that the left-hand side of (2.2) is invariant under 
transposition of the diagrams, which interchanges Frobenius coordinates. □ 

Set 

Fh k = Fs (k) = <p(h* k ), Fe k = Fs {lk) = <p(e%). 

The formulas of §1 together with the definition (2.1) lead to the following for- 
mulas for the FS functions. 

Generating series: 

1 + y Fhk =H{u) = 1 + y^ ( 2.3) 

OO j-, oo 

or, in terms of the super realization of A, 

y> Fh k (x;y) = tt 1 + y%/u 

11 1 - ^/ti 

1 v 2 ^ Fe k (x;y) = A 1 + 

Jacobi-Trudi and Nagelsbach-Kostka: 

Fs^ = det[Tj-i(Fh^-i + j)] 
= det[T 1 _ i (Fe M j_ i+i )]. 

or, in terms of the super realization, 

Fs^(x;y) = det[Fh^- i+j (x + j -l;y-j + 1)] 
= det[Fe^._ i+j (x - j + 1; y + j - 1)], 

where we abbreviate 

x + const = (xi + const, £2 + const, . . . ). 

Giambelli: 

Fs ll = det[Fs (p .\ q . ) ]. 

The next result is a generating series for the elements Fs( p \ q y It will be used in 
Theorem 2.4 and is of independent interest. 
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Theorem 2.3. We have 



l + (u + v) J2 



Fs (p\q) 



P j^o {u-\)...{u-^){v-\)...{v-^) 

= H(u)E(v), (2.5) 

cf. (1.5). Equivalently, 

Fs (p+i I q) + Fs (p I q+i) + (P + Q + l)Fs {p | g) = Fs (p | o)Fs (0 | g ) , p, q = 0, 1, 2, . . . , 

(2.6) 

c/. (1.4). 

Proo/. Since Fs( p \ q ) = v(S( p \ q )) and 

H{u) = <p(H*(u - §)), = ^(£> - §)), 

the relation (2.5) is equivalent to 



l + (u + v) 



s (p\q) 



p ^{u-\)...{u-^){v-\)...{v-^) 
= H*(u — \)E*{v — |) 



„* \ / OO „* 

(p|°) 1 ,r 



;(„-!)...(„- V ^ („-§)...(„- 2i±i) 
Writing 

w + „ = ( u - ^+1) + (v- %±±) + (p + q+l), 
we reduce the above formula to the system of relations 

S (p+l\q) + S *(p\ q+1) + (P + Q+ l) S (p| q) = S (p \ 0) S (0 | q) > £>,<? = 0, 1, 2, ... . (2.7) 

Note that this system is equivalent to (2.6). 

Thus, it suffices to check (2.7). This is a particular case of a Littlewood- 
Richardson-type rule for the s*-functions, found by Molev and Sagan [MS]. Here 
is an elementary derivation of (2.7). 

Let us expand the product s l P \o) s *o\ q ) a li near combination of the shifted 
Schur functions; such an expansion exists, because the shifted Schur functions form 
a basis in A*. By virtue of (1.4) we know the highest terms of this expansion, so 
we get 

S (p|0) S (0|q) = S (p+l\q) + S ( P \q + l) + Yl C (") S t , 

v. \h>\<p+q+l 

where c{v) are certain numerical coefficients. Let X be the set of those z/'s that 
enter this sum with nonzero coefficients c(u). We claim that X contains only the 
diagram (p\q); here we will use Proposition 1.3 and the fact that \v\ < p + q + 1. 

Indeed, let A be a minimal (with respect to inclusion) diagram in X and evaluate 
both sides at the point A = (Ai, A2, . . . ). On the right, the result is nonzero, because 
s^(A) ^ while all other terms on the right have zero contributions, because neither 
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a diagram v ^ A from X nor {p + 1 1 q) and (p \ q + 1) are contained in A. So, the 
result of the evaluation on the left is nonzero, too. This implies that A contains 
both (p | 0) and (0 | q), which is only possible for A = (p \ q). 
Thus, our expansion takes the form 

S (P I 0) S (0 | q) = S (p+1 | q) + S (p | q + 1) + COnSt ' S (p | q) » 

and it remains to find the constant. Evaluating both sides at A = (p \ q) we get 

_ »( P |(i)( A ) 3 (0|,)( A ) 

Applying Proposition 1.2 we get 

«^o.W= '" m SS k)) (P + t + 1) ^' = (P + t + 1)P ' 

Similarly, 

slo\q)W = (P + Q + 1 W- 

Finally, by (1.8), 

S (^)( A ) = ^mW = (P + 9+lW 
This implies const = p + 5 + 1, as was required. □ 

Our aim is to expand the FS functions in the S functions. 

Proposition 2.4. We have 

p <? 

F s (p \q) = ^2^2 c pp ,c ii' s {p' 1 <?') > ( 2 - 8 ) 

p'=0 q'=0 

where 

c -' = i 0, P ' > P , (2 - 9) 

/or anyp.p' = 0,1,2, 

(Note that c pp = 1.) 
Proof. By virtue of (2.5) and (1.5), 

^f(pk) 

( u _ 1 1 . . . ( u - 2£+i 



(u + v) v 



= H{u)E{v) -l = (u + v) 



00 

S(p\q) 



u p+l v q+l 
p,q=0 

Note that u + v can be written as (^ + ^)(^ • ^) _1 and that the algebra of formal 
power series in the indeterminates ^, ^ has no zero divisors. Consequently, we may 
divide both sides by u + v, which gives 

f_ s (p\q) = s (p\q) (9 in) 



p 

We need the following Lemma 
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Lemma 2.5. If a±, <22, . . . and &i, &2> • • • are number sequences, then for any 
p' = 0,l,..., 

/ip_ p / (61 , . . . , 6 p / + i ; — ai , . . . , — a p ) 



(u-6i)...(u- V+i) ^ (u - ai) . . . (u - a p +i) 

where ho = 1, /ii, /12, • • • denote the conventional complete homogeneous functions 
in the super realization of the algebra A. 

Proof of the Lemma. Indeed, we have 

! _ g d(p,j/) 



(u - 61) . . . (u - V+0 _ , - ai) • • • - ap+i) 



with certain coefficients d(p,p'). It is readily seen that d(p,p') is equal to the 
coefficient of -u _1 in the expansion of 

{u-ax) ...(u-a p ) e c ^i^ 



(u - bi) . . . (u - b p > +1 ) 

or, equivalently, to the coefficient of u p ~ p in the expansion of 

(u-ai)...(u- Op) 1 
(u - 61) ... (u - V+i)« p_p '~ 1 UJJ ' 

By (1.2'), the latter coefficient is exactly h p - p '(bi, . . . , £y+i ; — ai, . . . , — a p ). □ 

Returning to the proof of the proposition, let us apply Lemma 2.5 to 

a>\ = \i a>2 = 21 °3 = f > • • • > 61 = 62 = • • • = 0. 
Then, using the relation 

h (n n- _I _3 2p-i \ _ / ^p-p' /i 3 2p-i\ _ 

n p - p i \y, . . . , u , 2 , 2''''' 2 / — v ) c p-p' \ 2 ) 2 ' ' ' ' ' 2 / ~~ c pp'' 



we get 



Likewise, we have 



1 00 

' c pp' 



^ 00 

7+1 = ^2 



Cqq' 



whence 



_. 00 00 



Substituting this into the right-hand side of (2.10) we get 

p q 

Fs (p I «)) = c pp' c <?<?' I ?') ' 

p'=0 <j'=0 

which concludes the proof of Proposition 2.4. □ 
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Theorem 2.6. (i) We have the equality 

v 

summed over diagrams v which are contained in fx and have the same number of 
diagonal squares as fx. 

(ii) Write fi and v in Frobenius notation 

fx= (p 1 ,...,p d \q 1 ,...,q d ), p!> ••'>Pd>0, qi> •••> <?d > 0, 

v= (p'i,---,Pd\Q'i,---,Qd), pi > ••• >?/ d >0, q[> ■■■> q' d >0. 

Then we have 

c^ = det[c p . )P >] det[c q .^}, (2.11) 
where the determinants are of order d and the coefficients c pp / are defined by (2.9). 
Proof. Both Fs^ and s v can be expressed via the Giambelli formula, 

Fs/j, = det[Fs( p . \ Qj )], s I/ = det[s Cp /| g /)]. 

Consequently, it suffices to prove the claim of the theorem in the simplest case when 
\i is a hook diagram, fi = (p\q), which was done in Proposition 2.4. □ 

§3. Multiparameter Schur functions 

Let a = (ai)i e z be an arbitrary sequence of complex numbers. All the symmetric 
functions introduced in this section will depend on a. 

First, we define the multiparameter Schur functions hk- a which are indexed by 
one row diagrams (/c), i.e., certain analogs of the complete homogeneous functions. 
To do this we will employ generating functions which are formal series in it -1 , cf. 
(2.3): 

f^ i (u-a 1 )...(u-a k ) f^u k 

Clearly, we have 

hk-a = hk + lower terms. (3-2) 

This implies, in particular, that {hk- a }k=i,2,... is a system of algebraically indepen- 
dent generators of A. 
We agree that 

ho;a = 1, h-i. a = h- 2 - a = • • • = 0. 

We also need the following notation: for r G Z, let r r a be the result of shifting 
a by r digits to the left, 

(T r a)i = a i+r . (3.3) 

Now we are in a position to define the multiparameter Schur function indexed 
by an arbitrary Young diagram fi: 

s^ a = detlh^-i+j-Ti-ia] (3.4) 

where the order of the determinant is any number greater or equal to £(li), the 
number of rows in fi. Clearly, hk- a = S(k);a- 

Note that our definition (3.4) is a particular case of a very general concept of 
multi-Schur functions due to Lascoux [LI] (see also [L2] and [Ma3]). 
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Proposition 3.1. Multiparameter Schur functions as defined above satisfy the Gi- 
ambelli formula: 

s M;a = det[s {p .\ q . ) . a }, 

where the determinant has order d = d(fx) and pi, . . . ,Pd',Qi,- ■ ■ ,Qd denote the 
Frobenius coordinates of fx. 

Proof. This is immediate from Macdonald's result stated above as Proposition 
1.6. □ 

Note that if a = then s M;a = s M . For arbitrary a, it follows from (3.2) and (3.4) 
that 

s n;a = S[i + lower terms, 
which implies that the elements s M;a form a basis in A. 
Proposition 3.2. If ai = i — \ then s M;a = Fs^. 

Proof. Both functions can be given by a Jacobi-Trudi-type formula, see §2 and 
(3.4). Consequently, it suffices to prove that for = i — \ 

hk-a = Fhk, h k . T -r a = T r (Fhk). 

The first equality is immediate from the comparison of (3.1) and (2.3). Let us prove 
the more general second equality. By (3.1) and (3.3), 

1 + 7 = #(«)• 

f~[{u- ai_ r ) . . . (u - a k - r ) 

By our assumption on a, we have Substitute this in the latter 

expression and then replace « + rby«. Then we get 

! + Et ,=H(u-r). 

^ (it - a x ) . . . [u - a k ) 

The left-hand side is equal to 

hk-T~ r a 



1 + 



£J („-!)...(„- 



while the right-hand side, by virtue of Proposition 1.1 and (2.3), is equal to 

T r {Fh k ) 



{{u-\)...{u-m^)- 



This proves the second equality. □ 

Thus, the multiparameter Schur functions interpolate between the conventional 
S'-functions and the FS'-functions. 

19 



Remark 3.3. Following the general philosophy of symmetric functions, one can 
define multiparameter Schur functions indexed by skew diagrams X/fi by making 
use of the canonical comultiplication A : A — > A <g> A. We recall that A is specified 
by setting A(pk) = p\~ <S> 1 + 1 ®Pk, or, which is the same, it corresponds to splitting 
the collection of the variables into two disjoint parts: x = x' Ux" (and, in the super 
case, y = y' U y"). Then s A / M;a is defined by 

A(s A;a ) = ^S^a <g> S A/M;a . 

The result of the Appendix shows that s\/^. a vanishes unless \i C A, and an 
analogue of (3.4) holds: 

sx/^-a = det[/i A ._ M ._. +J , r ^- 3 -+iJ. 

□ 

As was emphasized in §0, our definition of the multiparameter Schur functions 
was suggested by Mo lev's work [Mo]. In the rest of this section we discuss the 
connection with [Mo]. 

Let A m)n denote the algebra of supersymmetric polynomials inro + n variables 
[BR], 4 and let Y mjn denote the set of Young diagrams not containing the square 
(m + 1, n + 1). It is well known that the conventional supersymmetric Schur poly- 
nomials s fI (xij . . . , x m ; yi, . . . , y n ), where fx ranges over Y mjTl , form a homogeneous 
basis in A m n . The algebra of supersymmetric functions can be identified with the 
projective limit of the graded algebras A m n as both m, n go to infinity. 

In [Mo], Molev introduced a family of multiparameter supersymmetric Schur 
polynomials, which depend on a and are denoted as 

s IJi (x 1 ,...,x rn /y 1 ,...,y n \a). (3.5) 

These polynomials are inhomogeneous, and their top degree homogeneous com- 
ponents coincide with the conventional supersymmetric Schur polynomials. This 
implies that the polynomials (3.5) form a basis in A m n . When a = 0, they reduce 
to the conventional supersymmetric Schur polynomials. 

Molev's initial definition is given in terms of a combinatorial formula. Next, he 
writes down generating series for the h- and e-functions and establishes a Jacobi- 
Trudi-type formula and its dual analog. He then gets a version of the Sergeev- 
Pragacz formula. His paper also contains a number of other results which will not 
be discussed here. 

Recall that a fundamental property of the conventional supersymmetric Schur 
polynomials is their stability: specializing x m = gives the supersymmetric Schur 
polynomial in (m — 1) + n variables with the same index, and similarly for y n = 0. 
Due to the stability property, one can define the supersymmetric Schur functions in 
oo + oo variables. It is pointed out in [Mo] that the polynomials (3.5) lose the stabil- 
ity property. Our observation is that stability is recovered if we restrict ourselves to 
the case m = n and slightly modify Molev's definition. The exact correspondence 
between Molev's polynomials and our multiparameter Schur functions is as follows: 

4 Here we say 'supersymmetric polynomials' in place of 'supersymmetric functions' in order to 
emphasize that one deals with finitely many variables. 
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Proposition 3.4. For any n and any n, 

s wo (xi, . . . , x n ; y u . . . , y n ) = s fJ ,(x 1 , . . . , x n /y u . . . , y n \r~ n a) 

Proof. Assume first that \x = (k), where k = 1,2,..., and write h k (- . . ) instead of 
s (fc )(...). By [Mo, (2.6)] we have 

j /i fc (a:i,...,a: n /yi,...,y n |a) _ (u + yi) . . .(u + y n ) 

£^ (u - a n+1 ) . . . (u - a n+k ) (u-x 1 )...(u-x n )' 

Replacing a by r~ n a we get, by (1.2') and (3.1), 

1 /ifc(xi,...,x n /yi,...,y n |r~ n a) _ (m + yi) . . . (m + y n ) 

(u-a 1 )...(u-a k ) (u-xi)...(u-x n ) 

tt / \t \ 1 , hk;a\Xl, ■ ■ ■ , X n ; J/i, ... , J/ n ) 

= H(u)(xi,...,x n ;yi,...,y n ) = 1+ > ? r r . 

(u-ai)...(u-a fc ) 

This implies that 

h k ;a(x 1 ,...,x n ;y 1 ,...,y n ) = h k (x u . . . , x n /y u . . . , y n |r" n a), (3.6) 

which is our claim for |it = (k). 

For a general ji we employ Molev's Jacobi-Trudi formula [Mo, (3.1)], which gives 

s M (xi, . . . , x n /yi, . . . , y„|a) = det[/i Mi _ i+J (xi, . . . , x n /yi, . . . , y n |r" J+1 a)], 

whence 

s M (xi, . . . , x n /yi, . . . , y n \r~ n a) = det[h^ i+j {x u . . . , x n /y 1 , . . . , y n |r" n_J+1 a)] 
= det[/i Ai ._ i+ j ;T i-i a (xi, . . . , x n \ y u . . . , y n )], by (3.6) 
= s M;a (xi,...,x n ;yi,...,y n ), by (3.4). 

This completes the proof. □ 

It should be emphasized that this correspondence fails when m ^ n. 

§4. Combinatorial formula 
We attach to a the 'dual' sequence a, given by 

a t = -a-i+i . (4.1) 

Note that in the FS case, we have a = a. 

Let 1! = {...,— !,...} stand for the set of proper half-integers. Cer- 

tain formulas will look more symmetric if we agree to label the terms of the sequence 
a by the half-integers. For this reason we introduce the alternative notation 

«' = (fle)eez', a' £ = a s+1/2 . (4.2) 
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In this notation, (4.1) takes symmetric form: 



(a)' £ = -a'_ e . 



(4.3) 



Note that in the FS case, a' e = e. 

Recall that a horizontal strip is a skew Young diagram containing at most one 
square in each column. Dually, a vertical strip contains at most one square in each 
row. More generally, we will deal with skew diagrams v of the following kind: there 
exists a skew subdiagram v\ C v such that v\ is a horizontal strip while vlv\ is a 
vertical strip ( equivalent ly, there exists v 2 Q v which is a vertical strip while v jv 2 
is a horizontal strip) . These are exactly skew diagrams v containing no 2 x 2 block 
of squares (equivalently, the contents of the squares □ G v are pairwise distinct). 
Such a diagram is called a skew hook 5 if, in addition, it is connected. Thus, a skew 
diagram with no 2 x 2 block of squares is a disjoint union of skew hooks. 

To each skew diagram v containing no 2 x 2 block of squares we attach a poly- 
nomial f u - a (u,v) in two variables u, v, of degree as follows. 

First, assume v is a skew hook. Consider the interior sides of the squares of the 
shape v: an interior side is adjacent to two squares of u; the total number of the 
interior sides is equal to \v\ — 1. To each interior side s we attach the coordinates 
(e, 5) of its midpoint, 6 and we write s = (e,5). Note that one of the coordinates 
is always half-integral while another coordinate is integral. Specifically, if s is a 
vertical side then s G Z', 5 G Z, and the ends of s are the points (e — 1/2, S) and 
(e + 1/2, 8); if s is a horizontal side then e G Z, 5 G Z', and the ends of s are the 
points (e,5-l/2), (e,8+l/2). 

For both vertical and horizontal sides, 5 — e G Z'. Using the notation (4.2), we 

set 



U., a (u,v) = (u + v) H (u-a' 5 _ £ ) fj (v + a' s _ e ). (4.4) 



For instance, if v = (4, 2, 2)/(l, 1) (see the figure below) then there are 6 squares 
and 5 interior sides with midpoints 



and we have 

f„-a(u, v) = (u + v)(u - a'_ 3/2 )(u - a' 3/2 )(u - a' 5/2 )(v + a'_ 1/2 )(v + a' 1/2 ) 
= (u + v)(u- a-i)(u - a 2 )(u - a 3 )(v + a )(v + ai) 

(On the figure, the interior sides and their midpoints are represented by dotted 
lines and bold dots, respectively.) 



5 Other terms: border strip, ribbon, see [Mai]. 

6 According to the 'English' manner of drawing Young diagrams, we assume that the first 
coordinate axis is directed downwards and the second coordinate axis is directed to the right. 



vertical interior 
sides s=(e,S) of v 



horizontal interior 
sides s = (e,<5) of v 



( 




(2,|), (1,|), (1,2), (1,3), 
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When v is an arbitrary skew diagram with no 2 x 2 block of squares, we define 
fu-,a(u, v) as the product of the polynomials attached to its connected components. 

Proposition 4.1. Let v be a skew Young diagram containing no 2 x 2 block of 
squares. Then 

fu;a(v,u) = f u >fr(u,v). 

Proof. This is immediate from (4.3) and (4.4). □ 

Let ii be a Young diagram. Recall that a seraistandard (or column-strict) tableau 
of shape fx is a function T{u) from the squares of \i to {1,2,...} such that the 
numbers T{u) weakly increase from left to right along the rows and strictly increase 
down the columns. For such a tableau T, the pull-back T _1 (i) C ii is a horizontal 
strip for any i = 1,2,..., see [Mai, 1.5]. Dually, for a row-strict tableau, each 
subset of the form T _1 (i) is a vertical strip. Now, we give the following definition: 

A diagonal-strict tableau of shape ii is a function T(n) from the squares of to 
{1, 2, . . . } such that the numbers T{u) weakly increase both along the rows (from 
left to right) and down the columns, and strictly increase along the diagonals j — i = 
const. We will also consider diagonal-strict tableaux with entries in {1, . . . , n}. 

Clearly, each subset of form T _1 (z) is a skew diagram with no 2 x 2 block of 
squares, i.e., a disjoint union of skew hooks. Thus, a diagonal-strict tableau T with 
entries in {1, . . . , n} may be viewed as a chain of Young diagrams, 

such that jjS % > j jU^ -1 ** has no 2 x 2 block of squares for each i = 1, . . . , n. 

Note that the definition of a diagonal-strict tableau also makes sense for a skew 
Young diagram \x. 

Given an ordinary or skew Young diagram \x and indeterminates x = (a^), y = 
(yi), consider the combinatorial sum 

r i>i 

summed over all diagonal-strict tableaux of shape fi. 
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By Proposition 4.1, we have 

£ M;a (y; x ) = y)- (4.6) 

Proposition 4.2. Assume Xi = yi = /or i > n. Then only tableaux T with 
entries in {1, ... , n} make nonzero contributions to the sum (4.5). 

Proof. Indeed, assume T takes a certain value i > n. Then for this i, the shape 
T _1 (z) C /U is nonempty. By the definition of the polynomials f u - a (see (4.4)), 
fr- 1 (i)-,a( x i > 2/i) contains the factor Xi + j/j, which is zero by the assumption. Con- 
sequently, the contribution of T is zero. □ 

Thus, under the above assumption, the sum (4.5) is actually finite. Note that 
the same holds under the weaker assumption that X{ = —yi for all i > n. 

Proposition 4.3. The sum (4.5) can also be defined by recurrence as follows: for 
any k < n, if A is a skew diagram, then 

S A; a(^l, • • • , Xk, Xk+l, ■ ■ ■ , X n ] 2/1, • • • , Vk, Vk+l, ■■■lUn) 

= 5^ E M;a(^i, ...,x k ;yi,..., yfc)E A/Al . a (a; fc+ i, . . . , x n ; y k+1 , ...,y n ), 

summed over skew diagrams [i contained in X, and 

fn;a(xi;yi), if fx contains no 2x2 block of squares, 
0, otherwise. 



£ M;a (o;i;yi) 



Proof. This is evident. □ 

We proceed with an alternative description of the sum (4.5). Consider the or- 
dered alphabet 

A = {1' < 1< 2' < 2 < ...} 

and call an A-tableau of shape \i any map T(-) from the set of squares of [i to the 
alphabet A such that: 

(*) The symbols T(n) weakly increase from left to right along each row and down 
each column. 

(**) For each i = 1,2, . . ., there is at most one symbol i' in each row and at 
most one symbol % in each column. 

This definition (as well as that of diagonal-strict tableaux) is suggested by the 
branching rules for the supersymmetric Schur polynomials, see [BR, §2], especially 
Theorem 2.15 in [BR]. Note also that the A-tableaux can be obtained via an appro- 
priate 'super' version of the Robinson-Schensted-Knuth correspondence, see [BR, 
§2] and [RS]. Strictly speaking, the version of the RSK correspondence given in 
[BR,§2] is related to another ordering of the alphabet A. However, the construction 
can be readily rephrased to handle our ordering. Actually, there are many different 
'super' versions of RSK, related to different shuffles of the primed and nonprimed 
indices. This fact was briefly pointed out at the bottom of page 125 of [BR]. For 
a detailed analysis, see [RS]. Note that the RSK correspondence implies various 
formulas for the enumeration of the A-tableaux. Finally, note that from another 
point of view, a 'super' version of the RSK correspondence was also discussed in 
[KV1]. 

On the other hand, A-tableaux were employed for shifted Young diagrams, in 
the combinatorial formula for the Schur Q-functions and their factorial analogs, see 
[Mai, III.8, (8.16')], [I]. 
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Proposition 4.4. The combinatorial sum (4.5) can be written as follows 



X„;a{x;y) = 



( \ 

) I (xt(d) - a c(D )) I J (V\T(n)\ + o c (n)) 
□ e/j 

\T(n) = l,2,... T(D) = 1',2',... / 



(4.7) 



summed over all A-tableaux of shape fx, where we use the notation \i'\ = i for 
i = 1, 2, . . . and c(n) denotes the content of z.e., if a = (i,j) then c(n) = j — i. 

Proof. Let z/ be a skew hook and i G {1,2,...} be fixed. Clearly, there exists 
exactly one diagonal-strict tableau T of shape v, with entries in {z}, and we claim 
that there exist exactly two A-tableaux T of the same shape, with entries in {i, i'}. 
Indeed, let be the squares of v written down in the order of increasing 

contents. Then T(Di) may be chosen arbitrarily, while for any r = 2, . . . , k, the 
value of T(n r ) is defined uniquely, according to whether the squares n r _i, u r lie 
in the same row or in the same column: in the former case, T{u r ) = i, and in the 
latter case, T(n r ) = i' . 

Next, if v is a skew diagram with no 2 x 2 block of squares then the same is true, 
with the only exception that the number of the T's is equal to 2 l , where / stands 
for the number of connected components of v. 

Now, let fx be an arbitrary (skew) diagram. To any A-tableau T of shape fi 
we assign a diagonal-strict tableau T by replacing each primed index i' by the 
corresponding nonprimed index i. Conversely, any diagonal-strict tableau T of 
shape fx can be obtained in this way from a certain (nonunique) A-tableau T. To 
get all such T's, we have to choose, for any nonempty diagram v = T _1 (i), an 
arbitrary A-tableau of shape u, with entries in {i,i'}, as described above. 

This means that the right-hand side in (4.7) can be written as a double sum, 
where the exterior sum is taken over the diagonal-strict tableaux T of shape and 
each interior sum is taken over all A-tableaux T 'over' a fixed T. It follows that 
the claim of the proposition can be reduced to the following one: let v by a skew 
Young diagram with no 2 x 2 block of squares; then 

/ \ 

f u . a (u;v)= 



11 ( u ~ a c(a)) ]} (v + a c (p)] 



\T(n)=i T(n)=i' / 

summed over all A-tableaux of shape z/, with entries in {1,1'}. 

Finally, without loss of generality, we may assume that v is a skew hook. Then, 
as was shown above, there are exactly two T's, so that the sum in the right-hand 
side of the last formula consists of two summands. On the other hand, the left-hand 
side is given by (4.4). Writing in that expression the factor u + v as the sum of 
u — a c and v + a c , where c = c(ni) is the smallest content, we split the left-hand 
side into two summands, too. Then the desired equality is readily verified. □ 

The following claim is a slight refinement of Proposition 4.4: 

Proposition 4.5. Assume Xi = yi = (or, more generally, Xi = —yi) for all 
i > n. Then, in the right-hand side of (4.7), one can take only tableaux T with 
entries in A n = {1' < 1 < • • • < n' < n}. 

Proof. Indeed, this follows from Proposition 4.3 and the proof of Proposition 4.4. □ 
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Note that the A-tableaux T, in contrast to the diagonal-strict tableaux T, are 
not consistent with transposition. In particular, the symmetry (4.6) is not evident 
from (4.7). However, in certain circumstances, it is more convenient to use formula 
(4.7) than formulas (4.4) and (4.5). 

Theorem 4.6 (Combinatorial formula). We have 

s^, a (x;y) = E^ a (x;y) 
where the right-hand side is given by (4.5) or (4.7). 
Proof. See the Appendix. □ 

Note that for the first time, an 'inhomogeneous' combinatorial formula probably 
appeared in [BL1], [BL2], see also [CL]. Other examples can be found in [GG], 
[Ma2], [Mo], [Okl], [001], [Ok2]. See also further references in [Ok2] to works 
by Knop, Okounkov, and Sahi about combinatorial formulas for interpolation Jack 
and Macdonald polynomials. 

Corollary 4.7 (Duality). We have 

^(^;a) = V;o- 

Proof. Indeed, this follows from Theorem 4.6 and (4.6). □ 
Set 

ek;a = S(ifc) ;a . 

By Corollary 4.7, 

u(hk;a) = Ck-a- (4.8) 
Corollary 4.8 (Generating series for e-functions). We have 

oo oo 

1 + E 7 ^V^T = = 1 + £ \ ( 4 - 9 ) 

Proof. Applying ui to both sides of (3.1) we get 

oo 

1 + Y ^ r = uj(H(u)) = E(u). 

Next, replacing a by a, we get (4.9). □ 

Corollary 4.9 (Nagelsbach-Kostka formula). We have 

Sfi-,a = det[e^_j_|_j ;r j-i a ] 

with the understanding that 

e -a = 1, e_i ;a = e_ 2 ;a = • • • = 0. 

Proof. This follows from (3.4), Corollary 4.7, (4.8), and the fact that (r r a)^ = 
T~ r a. □ 

Remark 4.10. Let us specialize a* = i — \. That is, take a' 5 _ e = S — e in (4.4), and 
a c{U) = C ( D ) — \ m (4.7). Then Theorem 4.6 turns into a combinatorial formula 
for the FS- functions. 
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§5. Vanishing property 
We fix a Young diagram \x and write it in the Frobenius notation, 

\i = (p 1 ,...,p d \q 1 ,...,q d ). 

Let A be an arbitrary diagram, 

A = (Pi, . . . , Pd I Qi, ■ ■ ■ , Qd)- 

Then /iQ means that d < D, pi < Pi, qi < Qi for i = 1, . . . , d. 
We define a collection of variables (x(X);y(X)) as follows 

x(X)i = a Pi+1 , y(X)i = a Qi+u 1 < i < D, 
x(X)i = y(X)i = 0, i > D. 

For instance, if = i — | then (x(A), y(X)) is exactly the collection of the 
modified Frobenius coordinates of A. 

Theorem 5.1 (Vanishing Theorem). 7/ /U <2 A then s fI - a (x(X);y(X)) = 0. 

Proof. We employ, in a slightly modified form, an argument due to Okounkov, cf. 
[Okl, proof of Prop. 3.8] and [001, second proof of Theorem 11.1]. 

Assume that s fl - a (x(X);y(X)) ^ and let us prove that AD//. 

Step 1. Let us prove that D > d. Employ for s fJ- - a (x(X);y(X)) the expression 
given by Theorem 4.6 and formulas (4.4) and (4.5). By Proposition 4.2, we can 
take in (4.5) only tableaux T with entries in {1, . . . , D}. On the other hand, the 
main diagonal in fx has length d and, by the definition of a diagonal-strict tableau, 
it is filled by strictly increasing numbers. Consequently, D > d. 

Step 2. By Corollary 4.7, the quantity s tl - a (x(X);y(X)) does not change under 
fj, i— > /j,', X i— > A', a i— > a. Consequently, to conclude that A D /j,, it suffices to prove 
that 

Pi> Pl ,...,P d >p d . (5.1) 
Write the coordinates Xi(X), yi(X), i = 1, . . . , D, in the reverse order, 

x = (xi, . . . , x D ) = (a Pc+ i, . . . , a Pl+ i), 
2/ = {vi,---,Vd) = {a QD+1 ,...,a Ql+1 ). 

Since s M;a (x; y) is symmetric in x and in y, we get 

s^-a{x]y) = s ll . a (x(X);y(X)) ^ 0. 

By Theorem 4.6, s^ a (x; y) is given by formula (4.7). Let us fix a tableau T which 
has nonzero contribution to the sum (4.7). For this T, we get, in particular, 

J] {x T(n) - a c(n) ) ^ 0. (5.2) 

T(D)=1,2,... 

We aim to prove that (5.2) implies (5.1). Note that, by Proposition 4.5, |T(n)| 
takes values in {1, . . . , D}. 
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Introduce the notation 



(k(l),...,k(D)) = (P D + l,...,P 1 + l), 

so that 

1 < k(l) < ■■■ < k{D). (5.3) 

In this notation, 

x r = a fc ( r) , r = l,...,D. 

Our argument will employ the following evident fact: 
(*) If, for a certain square □ G /z, T(n) is nonprimed then k(T(a)) ^ c(n). 
(Indeed, otherwise we would get 

^T(n) - o c (n) = °fe(T(n)) - a c (n) = 0, 

in contradiction with (5.2).) 

Step 3. Let T be the diagonal-strict tableau corresponding to T 
tion, T(n) = |T(n)| for any square □ e /z. Consider the squares □ 
first row in fa For these squares, 

T(l,l)<T(l,2)<...<T(l,/z 1 ). 

We claim that 

k(T(l,j))>j, j = l,..., fil . (5.4) 

Indeed, (5.4) is trivial for j = 1. Assuming that (5.4) is true for j < jo, let us 
check it for j = j + l. 

The numbers k{T{l,j)) weakly increase. Therefore, if k(T(l,j + 1)) < jo then, 
by the assumption, 

k(T(l,j )) = k(T(l,j + l))=j . 

It follows that T(l, jo + 1) is nonprimed (indeed, if T(l, jo + 1) were primed then 
T(l,jo) would be the same primed index, which contradicts to the definition of 
A-tableaux). Then for the square □ = (1, jo + 1) we get: T(n) is nonprimed and 
fe(T(n)) = jo = c(n), which is in contradiction with (*). Thus, we have proved 
(5.4) by induction. 

Step 4- Recall that the numbers T(n) = \T(n) \ take values in {1, . . . , D}. On the 
other hand, these numbers strictly increase as □ ranges over any diagonal in \i from 
top to bottom. Together with the inequalities (5.3) this implies that the numbers 
fe(T(n)) also strictly increase along diagonals. Consequently, for any square □ = 
E /j with i < j : 

k(T(i,j)) > k(T(i - 1, j - 1)) > • • • > k(T(l,j - i + 1)) > j - i + 1, (5.5) 

where, on the last step, we have used (5.4). 

Let us fix % = 1, . . . , d and set j = fa, which corresponds to the last square in the 
ith row of ji (since i < d, the assumption j > i is satisfied). Then j — i + 1 = pi + 1. 
On the other hand, by the definition, each number k( ■ ) is equal to a certain P r + 1, 
and recall that P\ > ■ ■ ■ > Pd, so that Pi + 1 > • • • > Pd + 1. Thus, (5.5) means 
that, for certain r\ < ■ • ■ < r^, 

P ri + 1 > • • • > P n + 1 > pi + 1. 

Since > z, it follows that Pj > Pi, which concludes the proof. □ 



. By its defini- 
= (l,j) of the 
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Theorem 5.2. Let fi be a Young diagram and let the symbols x(-), y(-) be as 
defined in the beginning of the section. We have 

s li ;a(x(ji);y{n))= JJ ( a M,-*+i - a j-Mp- ( 5 - 6 ) 

Note that 

a-m-i+i - a j-^ = a'^-i+i/2 - a'-^.+j-i/2 = a ^-i+i + a ^-i+i > 
where a' = (a' s ) was defined in (4.2). Hence, (5.6) can be rewritten as follows: 
s^ a (x(fi);y(fi)) 

= JJ (a'n-i+i/i ~ a'-^+j-i/i) = II ( a M,-*+i (5.7) 

These two expressions are symmetric with respect to /i <-> /i'. One more expression 
is given below in Proposition 5.4. 

Proof. As before, write in the Frobenius notation, = (p±, . . . ,pd \ qi, ■ ■ ■ , qd)- 
Let fx* C fi denote the shape which is obtained from fx by removing the border 
skew hook — the set of the squares G fi such that (i + 1, j + 1) ^ In the 
Frobenius notation, fx* is obtained by removing the Frobenius coordinates p\ and 

91, 

A»* = (P2,---,Pd|?2,---,9d)- 

Note that fi* is the smallest subdiagram in ii such that fij fi* has no 2 x 2 block of 
squares. 

Let x(fx) and y(/u) be obtained from x(fx) and y(fx) by writing the coordinates in 
the reversed order, 

x(fi) = (xx, ...,x d ) = (a Pd+1 , . . . , a Pl+1 ), 
y(p) = (yi,---,Vd) = (a qd +i,...,a qi+1 ). 

Then 

(xi, . . . , x d _i; yi, . . . , y d -i) = (x(fx*);y(fx*)). 
We use the fact that 

s^a{x(fi)]y(fi)) = s^ a (x(fi);y(fi)) = s wo (xi, . . . ,x d ;y~i, ■ ■ ■ ,y~d) 

and employ the combinatorial formula (4.5) and Theorem 4.6 to compute the latter 
expression, 

d 

s M;a (xi, • • • , x d ; yi, . . . , y d ) = JJ /r-i(»);o(*i, j/i), (5-8) 

r i=i 

summed over all diagonal-strict tableaux of shape /U. By Proposition 4.3 and The- 
orem 4.6, this can also be written as follows 

Sr,a(x(n);y(fj)) = ^ s v . a (x(n*); y(fi*))f^/ v - a (xd, Vd), 

29 



summed over all subdiagrams v such that \xjv contains no 2 x 2 block of squares. 

By Theorem 5.1, s u - a (x(/j,*); y(/J,*)) = unless v is contained in /i*. By the 
minimality property of //* mentioned above, this means that v = fi* . Repeating 
this argument we conclude that in the sum (5.8), there is only one tableau T = Tq 
with (possibly) nonzero contribution: the chain of diagrams corresponding to Tq 
looks as follows: 

••• C (At*)* C A** C fi. 

Finally, let us check that the contribution of Tq is indeed given by formula (5.6). 
Let a*[1] C /i stand for the first diagonal hook in \i, 

fi[l} = {(l,j)eti}u{(i,l)efi}. 

We will prove that 

U/n-,a(xd,yd) = II Kii-i+l/2 -a'-p'.+j-i/i)- ( 5 - 9 ) 

Moreover, the same argument will prove that 

U*/W;a(Xd-l, Vd-l) = II Kii-i+l/2 ~ a '-/i;.+j-l/2)» 

(i,j)ev[2] 

where a* [2] stands for the second diagonal hook, and so on. This will imply (5.6). 
By definition (4.4), 

ft,/n*;a{xd,y~d) = {u + v) fj (u-a' Sk _ £k ) JJ {v + a' Sk _ Sk ), (5.10) 

Sfe is vertical Sfc is horizontal 

where (ek,$k) are the midpoints of the interior sides Sk of the shape \ij \s* and 
u = Xd = a Pl +i = a' Pl+ i/2> v = Vd = a qi +i = — a'-( qi+ i/2) ■ 

We establish a bijective correspondence s <-> between the sides s = Sk and 
the squares (z, j) G , except the diagonal square (1, 1), as follows: 



8 = (e, 5) <-> = 



(1, 5 + 1), if s is vertical, 
(e + 1, 1), if s is horizontal. 



Note that if = (1,5 + 1) then e+ 1/2 = //J, and if (z,j) = (e + 1,1) then 
5 + 1/2 = A*i- It follows that, under the above correspondence, the contribution of s 
to (5.10) coincides with the contribution of to (5.9). As for the factor (u + v) 
in (5.10), it coincides with the contribution of the square (1, 1). This proves (5.9) 
and concludes the proof of the theorem. □ 
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Corollary 5.3. If the numbers ai are pairwise distinct then s M;a (a;(//); y(/J.)) ^ 0. 

Proof. Recall a well-known claim: the sets {fii — i + l}^i and {j — n'^f^i do not 
intersect (and, moreover, their union is the whole Z), see, e.g., [Mai, I, (1.7)]. It 
follows that, under the assumption on a, all the factors in the product (5.6) do not 
vanish. □ 

In the special case ai = i — \ , Theorem 5.2 means that for any Young diagram 
fi = (p 1 ,...,p d \q 1 ,...,q d ), 

Fs^(p 1 + ±,...,p d +±;q 1 + ±,...,q d +±)= (^ - i + ^ - j + 1), (5.11) 



which is equal to 



n mu)=^. 



the product of the hook lengths. There exist other useful expressions for this 
product, in particular, 

d d 
l<i<k<d 

(see, e.g., [BR, 7.14.1], where one must take k = I = d, or [01]). The next proposi- 
tion provides a generalization of the last identity. 

Proposition 5.4. For any Young diagram fx = (pi, . . . ,p d \ qi, ■ ■ ■ , qd), the product 

= n Ki-i+1/2 - a -n' j+ j-i/2) = n ( a Mi-<+i+s;-j+i)> ( 5 - 12 ) 

which gives the value of s fX . a (x(fj,);y(n)), is equal to 

d d 
II ( a Pi+i - «i) • - -(api+i - a pJ(a 9i +i - «i) •• - a«) II ( a P ,+i + 

*=1 M = l 

II ( a p t +i - a p fe +i)( a «i+i - ag fe +i) 
i<i<fc<d 

(5.12') 

Note that in the special case a; = % — \ , this coincides with the right-hand side 
of (5.11). 

Proof. The part of the product (5.12) corresponding to the squares € £t with 
< <^ exactly coincides with the second product in the numerator of (5.12'). 
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It follows that the equality of (5.12) and (5.12') can be reduced to the following 
identity 



JJ (a^-i+i - ajw.) • JJ (V-i+i -S *-A*i) 



i<d,j>d 



j<d,i>d 



d d 

11 ( a Pi+i - ai) • • • (a Pi +i - a Pi ) II - ai) • • • - a ft ) 

i=l i=l 



n ( a Pl +i - a p fc +i) 

Ki<fe<d 



II -Oflfk+i) 

KkKd 



By symmetry, it suffices to prove the identity 



II (a Pi +i - ai) • • • (a Pl+ i - a Pi ) 



i<d,j>d 



n ( a Pi+i 
i<i</c<d 



a p fe +i) 



(5.13) 



which reduces to the following claim: for any diagram and any i = 1, . . . , d, 

{l,..., fH -i} = {fij - j + l}j =i+1 U {j - fi^ =d+1 , (5.14) 

a disjoint union of two sets. 

By virtue of the combinatorial fact mentioned in the proof of Corollary 5.2, the 
two sets in the right-hand side of (5.14) are indeed disjoint. 

Next, each of the two sets in the right-hand side of (5.14) is contained in the set 
from the left-hand side. Indeed, the minimal element of the first set is — d+1 > 1 
and the maximal element is fMi+i — i < Hi — i. Similarly, in the second set, the 
minimal element is d+l—fx' d+1 > d+l—d = 1 and the maximal element is /i'^. < 
Hi — i. Finally, the total number of elements in both sets is (d — i) + (pi — d) = fa—i, 
which is equal to the number of elements in the left-hand side. This proves (5.14) 
and concludes the proof of the proposition. □ 

The next two results are similar to the characterization theorems for shifted 
Schur functions, cf. [Okl], [001, §3]. 

Theorem 5.5 (Characterization theorem I). Let \i be an arbitrary Young diagram 
and 

D(fu) = {A G Y | |A| < \n\,X^ fu}. 

Assume that the numbers ai are pairwise distinct. Then, as an element of A, 
■V;a ^ s un, i>q ue ly determined by the following two properties: first, its top degree 
homogeneous component is the Schur function s^; second, it vanishes at (x(A),y(A)) 
for all A G D(n). 

Proof. Assume F is a symmetric function with the same two properties. Then there 
exists an expansion of the form 



F = s M;a + ^ c ( z/ ) s ^;« 
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with certain numerical coefficients c{v). We must prove that these coefficients are 
actually equal to zero. Let X stand for the set of z/'s with c(y) 7^ 0. Assume 
that X is nonempty and choose a minimal diagram A G X with respect to the 
partial ordering by inclusion. We get F(x(X);y(X)) = s M;a (x(A); y(X)) = 0, because 
A G D(pt). On the other hand, since A is minimal, we also have s u - a {x{X); y(X)) = 
for any v G X \ {A}, because v is not contained in A. But sx- a (x(X);y(X)) 7^ 0, 
because of Corollary 5.3. This leads to a contradiction. □ 

Theorem 5.6 (Characterization theorem II). Let [i be an arbitrary Young diagram 
and D(n) be as above. Assume again that the numbers ai are pairwise distinct. 
Then, as an element of A, s^ [a is uniquely determined by the following three prop- 
erties: first, its degree is less than or equal to second, its value at (x(fx);y(fx)) 
is given by formula (5.6); third, it vanishes at (x(A),y(A)) for all X G D(fi). 

Proof. Let F G A possess the same three properties. Then 

F= c(v)sv,a 

"6»(c)U{f} 

with certain numerical coefficients ciy). The same argument as above proves that 
c(v) = for all v G D{pt), so that F is proportional to s M;a . Then the second 
property implies that F is exactly equal to s^- a . □ 

In the particular case of the Frobenius-Schur functions, the results of this section 
are equivalent to similar claims for the shifted Schur functions contained in [Okl], 
[001, §3]. Interpolation of arbitrary polynomials in terms of Schubert polynomials 
is discussed in [L2]. A very general scheme of Newton interpolation for symmet- 
ric polynomials is developed in [Ok2]. In the latter paper one can also find the 
references to earlier works by Knop, Sahi, and Okounkov. 

§6. Sergeev-Pragacz formula 

Recall first the conventional Sergeev-Pragacz formula. Let m,n be arbitrary 
nonnegative integers and let fx be an arbitrary diagram not containing the square 
(m+ l,n+ 1). Then 

f v _ T /we e m xe n £ ( w ) ■ ■ ■ , s m ; 3/1, • • • , y n )} 

s, (Xl ,..., x m ; Vl ,..., y n ) - v{x ^ Xm)v{yu mmmfyn) 

(6.1) 

where 



m n 



fv(x 1 ,...,x m ;y 1 ,...,y n ) = l)++m *• JJ yf 3 j)++n 3 • JJ (xi + yj). 

i=l J = l i<m,j<n 

(6.2) 

Here & m x & n is the product of two symmetric groups acting on polynomials 
in xi,...,x m and yi,...,y n by permuting separately the x's and y's; for w = 
(wi, W2) G &m x & n , the symbol e(w) means sgn(wi) sgn(w 2 ); 

V(x 1 ,...,x m ) = Yl ( x i~ x k), V(y u ...,y n )= JJ (yj-yi), 

l<i<k<m l<j<l<n 
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and 

(k) + = max(fc, 0). 

Note that for n = the formula (6.1) reduces to the classical formula (0.6) for 
the Schur polynomial s^(xi, x m ). Proofs of (6.1)-(6.2) can be found in [PT], 
[Mai, 1.3, Ex. 23]. 

Our aim in this section is to establish an analog of the formulas (6.1)-(6.2) for 
the multiparameter supersymmetric Schur polynomials depending on a sequence 
of parameters a = (a^). According to our basic principle, we will deal with equal 
number of the x's and j/'s. 

Theorem 6.1. Let n = 1, 2, . . . and let \x be an arbitrary Young diagram such that 
d = d(n) < n. Then, in the above notation, 

E w66 „x6 71 £ ( w ) M-W^i, ■■■,x n ;y 1 ,..., y n )] 
V(x 1 ,...,x n )V(y 1 ,...,y n ) 

(6.3) 



S(j,;a(-Kl ) • • • ) ^ni Vl > • • • > Un) 

where 



/ M;a (x 1; r„:y : y n ) f[(x t \ a)"-**'""^* { Vi | a)^^ )+ 



i=l 



and 



(x|a) 7 



i=d+l i,j<n 



(x - ai) . . . (x - a m ), m > 1, 
0, m = 0. 



Note that for the sequence a = formulas (6.3)-(6.4) reduce to formulas (6.1)- 
(6.2) with m = n. Indeed, it suffices to check that / M;a reduces to / M . To do this, let 
us compare (6.2) and (6.4). The last product in both formulas is the same. Then 
we remark that 



(jii - n)+ + n-i 



Hi - i + (n - i = l,...,d, 

n — i, i = d + 1, . . . , n, 



and likewise for \i' , which implies the equality = / M;a | _ - 

Proof. The proof of Theorem 6.1 is divided into three lemmas. 

Lemma 6.2. The expression (6.3) is a supersymmetric polynomial. 

Proof. One may argue exactly as in the proof of Proposition 2.3 in [PT] (see also 
[Mai, 1.3, Ex. 24]). For the reader's convenience we present the argument. 

Obviously, (6.3) is a polynomial, which is separately symmetric in the x's and 
j/'s. Let us verify the cancellation property: assume that X{ = t = —yj for certain 
indices i, j, and let us prove that (6.3) does not depend on t. Consider the expression 
(6.3) as a function in t. This is a rational function, which is actually a polynomial. 
The degree of the denominator is exactly 2n — 2, so that it suffices to prove that 
the numerator has degree less than or equal to 2n — 2. Next, it suffices to prove 
the latter claim for the polynomial (6.4). 
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Without loss of generality, one may assume i < j. Consider three cases. 

First case: i < j < d. Then G fi, so that (6.4) contains the factor (xi + Vj)- 
This factor vanishes when xi = —jjj, consequently, (6.4) vanishes identically. 

Second case: i < d < j < n. The degree of the expression (6.4) with respect to 
t is less than or equal to the sum of three terms: 

(fii-i+(n- + ) + (n- j) + (pi + fJLj). 

Again, if G fi then (6.4) is identically equal to 0, so that one may assume 

[i- This means that fii < j — 1 and fi'^ < % — 1; in particular, fii < n. It 
follows that the above sum reduces to 

(n-i) + (n-j) + (jii + fij), 

which is < 2n — 2, because Hi + ji', < i + j — 2, as was mentioned above. 
Third case: d + 1 < i < j < n. Then the degree in question is 

(n-i) + (n- j) + (m + fji'j) < 2n - i - j + {j - 1) + (i - 1) = 2n - 2. 

This concludes the proof of the lemma. □ 

Let us temporarily denote the right-hand side of (6.3) by s'^. a (xi, . . . , x n ; yi, . . . , y n ). 
Lemma 6.3. We have 

s H;a( x l-i ■ ■ ■ i x n',yi, ■ ■ ■ , Vn) \ Xn=yn=0 

( 0, d = n, 

\ s' li . a (x 1 ,...,x n - 1 ;y 1 ,...,y n - 1 ), d<n-l. 

Proof, a) If d = n then the polynomial / M;a (^i, • • • , x n ; yi, . . . , y n ) is divisible by 

n 

11 ( x i + yj)> 

and the same holds for its transformations by elements w. Since the above prod- 
uct vanishes when x n = y n = 0, the whole expression s'^. a (xi, . . . , x n ; yi, . . . , y n ) 
vanishes, too. 

b) Next, let us assume that d < n — 1 and let us prove the following claim: 
fna(x 1 ,...,x n ;yi,...,y n ) \ x . =y . =0 = unless i = j = n. 

Indeed, vanishing holds if G fx, because then the polynomial in question 
is divisible by Xi + yj. Assume i < d. If /ii > n then G fx, which implies 

vanishing. If jii < n then (n — fii)+ > 0, so that vanishing holds thanks to the 
factor x\ n in the expression (6.4). Similar argument also holds when j < d. 

Thus, we may assume that both i > d and j > d. Then the expression (6.4) 
contains the factor x^^y™ -3 , so that the only possibility of nonvanishing may occur 
for i = j = n. 
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c) The numerator of (6.3) is an alternate sum of terms indexed by couples of 
permutations w = (wi, 102). If w does not fix the indeterminates (x n , y n ), which are 
specialized to zero, then the corresponding term vanishes after the specialization 
because of the claim b) proved above. Consequently, only terms indexed by elements 
w fixing (x n , y n ) may give a nonzero contribution. Note that these are actually 
elements of the group & n -i x &n-i- 

On the other hand, note that 

V(x u . . . , x n _i, 0)V(yi, . . . , y n _i, 0) = V(x u . . . , x n _i)V(yi, . . . , y n -i) 

x xx . . .x„_iyi . . .y„_i • 

d) Thus, it remains to check that 

;yi,...,y n _i,0) = / M;a (^i, • • • , Sn-i; J/i, • • • , Vn-i) 
x xi . . .x n _iyi . . .y n _i . 

Let us examine the behavior of the expression (6.4) under the specialization x n = 
y n = 0. The right-hand side of (6.4) consists of three products. The second 
product, which is equal to 

n 

n 

i—d+l 

turns into the similar expression for n — 1 multiplied by 

n-l 

n Xiyi - 

i=d+l 

Consequently, we must prove that the remaining expression in (6.4), which is equal 
to 

n(xda)^-x (n - Mi)+ ^ia)^-^ (n " M:)+ - n 

i=l 

turns into the similar expression for n — 1 multiplied by 

d 

i=i 

This is equivalent to the following claim: for any % = 1, . . . , d, 

(n - +e t = (n - 1 - + 1, (6.5) 

where 

1, (i,n) e fi, 
0, (i,n) i fi, 
and similarly for 

Consider two cases: \ii > n and Hi < n — 1. In the former case, (n — = 
(n — 1 — = and £j = 1, so that (6.5) holds. In the latter case, in — fj>i) + = 
{n—l—jii) + + l and £j = 0, so that (6.5) is again true. This concludes the proof. □ 

By Lemma 6.2 and Lemma 6.3, there exists an element s'^. a G A, of degree | /x j 
and such that the corresponding supersymmetric polynomials in n + n variables 
are given by the expressions (6.3) when n > d and vanish when n < d. We aim at 
proving that s'^. a = s M;a by making use of the characterization theorems from §5. 
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Lemma 6.4. The element s'^. a G A defined above satisfies the vanishing condition 
of Theorem 5.5, i.e., 



s 'u:a( x ( x );y( x )) = unless /I CX. 



Proof. We will see that the vanishing in question is ensured by the product 

d 



i=i 

entering the expression (6.4). 

Write both diagrams in the Frobenius notation, 

/i= (pi,...,Pd|5i,---,5d), A= (Pi,...,Pd\Qi,---,Qd), 
and recall that 

x(X)i = a Pi+1 , y(X)i = a Qi+1 , 1 < i < D, 
x(X)i = y(X)i = 0, i>D. 

By the definition of s^. a , vanishing holds if D < d, because x(X) and y(X) have at 
most D nonzero coordinates, so that we may assume D > d. 
Abbreviate 

Xi = x(X)i = a Pi+1 , yi = y(X)i = a Qi+1 , 1 < i < D. 

It suffices to prove that for any w = (w±, w 2 ) G &d x 6d, 

d 



w 



Jlix^aY^y^ay 



i=i 



= 



unless Pi > pi and Qi > qi for all i = 1, . . . , d. By the symmetry x <-> y, a <-> a, it 
suffices to prove that for any w\ G &d, 



no* i *y 



i=l 



= 



(6.6) 



unless Pi > pi for all i = 1 , . . . , d. 

The left-hand side in (6.6) has the form 



f [\(ap H+1 | a) p * = n( a ^+! - a i) • • • ( a ^+i - a pJ > 



i=l 



where j\, . . . , jd is a certain <i-tuple of pairwise distinct indices from {1, . . . , D}. If 
(6.6) does not hold then 

P h >Pi,...,P jd >p d . (6.7) 
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Let us check that this implies 



Pi>Pi,---,Pd> Pd- 



(6.8) 



Indeed, recall that 



Pi>--->P d , 



Pi> ■■■> Pd- 



Together with (6.7) this implies that among the numbers Pi,..., Pa, there is at 
least one number > pi (namely, -P/J, at least two numbers > P2 (namely, Pj 1 , Pj 2 ), 
and so on, which implies (6.8). □ 

Now we are in a position to prove the equality s'^. a = s^- a . Comparing the 
formula (6.3)-(6.4) defining s'^. a with the Sergeev-Pragacz formula (6.1)-(6.2) for 
the Schur function we see that the top degree homogeneous component of s'^. a 
coincides with s M . By Lemma 6.4, s'^. a possesses the same vanishing property 
as s M;a . Consequently, by Theorem 5.5, s'^. a = s M;a . Note that in Theorem 5.5, 
the numbers ai are required to be pairwise distinct but one may attain this by 
making use of the continuity argument, because both s^. a and s^. a depends on the 
parameters continuously. 

An alternative way is to apply Theorem 5.6. Then we must verify for s'^. a the 
three properties listed in the statement of this theorem. The first property (control 
of degree) is obvious. The third property (vanishing) is ensured by Lemma 6.4. 
The second property (required value at (x(n);y(/j,))) is verified as follows. From 
the proof of Lemma 6.4 one sees that in formula (6.3) applied to the variables 
(x(fj,);y(fjL)), all terms with w ^ e are zero. Next, it is readily seen that the term 
with w = e leads to the expression (5.12'), which, by Proposition 5.4, coincides 
with s^ a {x{ji); y(fjL)). 

This concludes the proof of Theorem 6.1. □ 

The following result is an analog of the Berele-Regev factorization property for 
the supersymmetric Schur functions [BR], [Mai, 1.3, Ex. 23]. 



Corollary 6.5. Let fx = (pi, ■ ■ ■ ,Pd\qi, • • ■ ,Qd) be a Young diagram of depth d. 
Then 



Proof. We apply formulas (6.3) and (6.4) for n = d. Then the last product in 
(6.4) coincides with the product above, which is invariant under permutations w. 
Consequently, the alternate sum in (6.3) becomes the product of two determinants. 
Note also that for each % = 1, . . . , d, we have Hi — i = pi, (n — = — /-*«)+ = 0' 
and, similarly, ^ — i = qi, (n — /^)+ = 0. □ 



Formulas (3.1) and (4.9) yield the generating series for the one-row and one- 
column multiparameter Schur functions; let us rewrite them in slightly different 
notation: 



det[(xi\a)Pi]t J=1 
V(xi,...,x d ) 



det[(^|a)^ J=1 
V{yi,...,y d ) 



d 



s^ a (xi,...,x d ;yi,...,yd) 



■ n fa + yj) ■ 



7. Transition coefficients 



00 



00 




(7.1) 



oc 



oo 
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The next proposition yields the generating series for the hook functions S( p | g ). a ; 
this series is an element of A[[*u _1 , i> -1 ]]. 

Proposition 7.1. We have 

CO 

1 + („ + „) y ^' q)5 ° +1 =H(u)E(v). 

v ; ^ (u \a)P +1 (v \a)i +1 v ; v ' 

p,q=0 V 1 ' 



Proof. This is a generalization of Theorem 2.3, and we will argue as in the proof of 
that theorem. By virtue of (7.1), the equality in question is equivalent to 

CO / CO \ / CO \ 

Using the identity 

U + V _ (u - Og+i) + (f - flg+l) + (flg+l + flg+l) 

(u | a)P +1 (v | a)« +1 ~~ (it | a)P +1 (^ | a)^ 1 

1 1 a p+ i + a g +i 

~~ (w|o)p(v|o)9 +1 (u\a)P +1 (v\a)i (u \ a)P +1 (v \ a)i +1 

we reduce this to the following system of relations, where p, q = 0, 1, . . . : 

S(p+1 | g);o + S (p | <J+l);a + («p+l + a q+ l)S( p | q ) ;a = S( p | );aS(0 | q);a • 

By virtue of (1.4), the expansion of the product S( p | o) ; a s (o | q);a into a linear 
combination of the functions s^ ;a has the form 

S(p\0);aS(0\q);a = S( P +l\q);a + S(p | g+l);a + ^ c(zv)Sj, ;a . 

Let X be the set of those z/'s which enter this sum with nonzero coefficients c(u). 
We claim that X contains at most the diagram (p\q); here we will use the fact that 
\u\ < p + q + 1. Indeed, let A be a minimal (with respect to inclusion) diagram 
in X and let us evaluate both sides at (x(X);y(X)). On the right, the result is 
nonzero, because s\ ]a (x(X);y(X)) ^ 7 while all other terms on the right have zero 
contributions, because neither a diagram v ^ X from X nor (p + l\q) and {p \ q+ 1) 
are contained in A. So, the result of the evaluation on the left is nonzero, too. This 
implies that A contains both (p | 0) and (0 | q), which is only possible for A = {p \ q). 
Thus, the above expansion takes the form 

S(p | 0);aS(0 \ q);a = S(p+1 | q);a,S( P \ q+l);a + Const S( p | q ). a . 

Evaluating both sides at 

(x(p\q);y(p\q)) = (a p+ll 0, 0, . . . ; a q+u 0, 0, . . . ) 

7 Here we tacitly assume that the sequence a = (aj) has no repetitions in order to apply 
Corollary 5.3. To cover the case when repetitions are present, one can use the continuity argument. 
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we get 

congt _ 3( P | o);a(x{p \ q)]y(p | q)) s {0 \ q) . a (x(p \ q);y(p \ q)) 

S( P \q)(x(p\q);y(p\q)) 

The right-hand side can be readily evaluated by making use of Corollary 6.5. 
This gives the desired result: const = a p+ i +a q+ \. □ 

Let b = (6j)igz be another sequence of parameters. We aim at expressing the 
functions s M;a through the functions s v ^. In particular, we are interested in the ex- 
pansion on the conventional supersymmetric Schur functions s u , which correspond 
to b = 0. 

For p > p' > set 

c pp '(a,b) = hp-p'ib!, . . . ,b p > +1 ; -ax, . . . , -a p ), (7.2) 

where h p - p ' is the conventional supersymmetric h- function of degree p — p' . In 
particular, 

c pp '(a,0) = h p -p'(0, ...,0;-ai,...,-a p ) = (-l) p ~ p e p _ p '(ai, . . . , a p ), (7.3) 

cf. (2.9). Note that c pp {a,b) = 1. 
Proposition 7.2 (cf. Proposition 2.4). H^e /mt>e 

p <? 

S(p\q);a= Yl ^ Cp p /(a, 6) C„/(a, 6) S(p/ | g / );b . (7.4) 
p'=0 q' = 

Proof. We have the identity 

oo oo 

>Z^ q {u \ a)P+ i {v \ a)q+ i ^^ o(u | 6)p+1(v | 6)g+1 ' 

because, by Proposition 7.1, both sides are equal to H(u)E(y) — 1. Then we argue 
exactly as in the proof of Proposition 2.4. □ 

Theorem 7.3 (cf. Theorem 2.6). Let a = (a^gz and b = (bi)i e z be two sequences 
of parameters. In the expansion 

Sp,-, a = ^c>(a,&) s v . b (7.5) 

the coefficients c^ u (a, b) vanish unless v C fj, and d{v) = d(ff). 

Assume d{v) = d(fi) and write both diagrams in the Frobenius notation, 

/I = (p 1 ,...,p d \q 1 ,...,q d ), v = (p' 1 ,...,p' d \q[,...,q' d ). 

Then 

Cp V (a,b) = det[c PuP >.(a,b)] det[c quq '.(a,b)], (7.6) 

where the determinants are of order d and the coefficients in the right-hand side 
are defined by (7.2). 

Proof. The same argument as in the proof of Theorem 2.6 reduces the desired claim 
to the special case d = 1, which was the subject of Proposition 7.2. □ 
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Appendix by Vladimir Ivanov: Proof of the combinatorial 
formula for multiparameter schur functions 

The aim of this appendix is to prove Theorem 4.6, which is restated below as 
Theorem A. 6. Here we are using a slightly different notation for the combinatorial 
sum, the /i-functions and the s-functions. 

Definition A.l. Suppose (cii)i e z is an arbitrary sequence of complex numbers. 
For fc = 0,1,2,..., set 

k 

(x | a) k = JJ(x-ai). 

i=i 

Definition A. 2. Fix n = 1,2, .. . and denote by A n the ordered alphabet {1' < 
1 < 2' < 2 < • • • < n' < n}. Put = \i\ = i for % = 1, 2, . . . , n. 

Definition A. 3. Fix a skew Young diagram A/ ' ji. A map T : X/fj, — > A n is called 
a tableau of shape A//i and of order n if the following conditions hold: 

(1) T(i,j)<T(i,j + l); 

(2) T(iJ)<T(i + lJ); 

(3) for each % = 1, 2, . . . , n, there is at most one symbol i' in each row; 

(4) for each i = 1, 2, . . . , n, there is at most one symbol i in each column. 

Let us denote by Tab(A/ fJ,,n) the set of all tableaux of shape X/ ji and order n. 

Definition A. 4. Suppose that x±, X2, ■ ■ ■ , yi, J/2, • • • are variables, (ai)i E z is an 
arbitrary sequence, /iCA are Young diagrams. Set 

^\/ij,\n{x;y | a) 

Yl II ( X T(P) -°c(n)) IJ (VT(n) + a c (n)), 

T(n)=l,2,...,n T(D) = l',2',...,n' 

where c(i,j) = j — i, cf. (4.7). 
Definition A. 5. Let /j, C A. Set 

s\/ fJ ,(xi,X2,...,x n ]y 1 ,y 2 ,...,yn I a) = 

det[/i Ai - w +j-;(^i, ^2, • • • , x n ; yi, y 2 , • • • , y n \ r^~ J+1 a)] 1 < iJ < e{x) , 



where the polynomials hk(xi, . . . , x n ; yi, . . . , y n \ a) are given by the generating 
series 



1 h k (xi,...,x n ;yi,...,y n | a) = -pr -u + ^ 

f-f (m I a) k u — X{ 

k=l 2=1 



= if k < 0, and ho = 1, cf. (3.4). 
Theorem A. 6. 



s\/p(x 1 ,x 2 , ■ ■ ■ , x n ; y u y 2 , . . . ,y n \ a) = E A/M | n (a;; y | a). 
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Proof. First let us prove this claim when A = (k),p = 0. Then S\/^(x;y | a) = 
h k (x;y | a) and our claim is equivalent to the following branching rule (see Propo- 
sition 4.3): 

hk(xi,...,x n -i,x n ;yi,...,y n -i,y n \ a) = h k {x\, . . . , x n -i; yi, . . . , y n -\ \ a) 

k—l k—l—r 

+ {x n + y n )^2h r (x 1 ,...,x n - 1 ;y 1 ,...,y n - 1 \a) JJ (x n -a k -t)- 

r=0 t=l 

Let us denote h k (x 1 , . . . , x n _i, x n ; y 1 , . . . , y n _i, y n \ a) by h k \ n {x;y \ a). From 
the definition of h k we obtain 

1 | h k \ n (x; y \ a) _ yr u + yi _ u + y n y fe fe | n _i(a;; y | a) u + y n ,^ ^ 

^ (u \ a) k *-\u — Xi u — x n ^ (u I a) k u — x n 

k =i «=i fc=i 

For any k = 1, 2, . . . we have 

^ = l + (x + y)J2 , 1 fc w • A -2 

U — X ' "• I T K n\ r 



This follows, for example, from Molev's results [Mo, Prop. 1.2, Theorem 2.1] if we 
put m = n = 1. Using (A.l) and (A. 2), we obtain 

1 h k]n (x;y \ a) _ u + y n /i fc | n _ 1 (a;; y | a) it + y n 



(it I a) fc u — x 7 

fc=l 



(*£«. | 


a)*" 1 


(« 


1 a) fc ' 



^|n-i(^;y 


a) + (x„ + y n ) Er=d 


/i r | n _i(x;y 


a ) llt=i r ( x n - a fc _ t ) 


(«| 


o) fc 



fc=l 

Therefore, when A = (k) and fj, = 0, the assertion of the theorem is proved: 
h k \ n (x;y I a) = j | (x T(D) - a c(n) ) (yr(p) + a c (n))- 

TeTab((fc),n) aeX/n D6A//J 

T(n)=l,2,...,n T(n)=l',2',...,n' 

(A.3) 

Now suppose that X/ fx is an arbitrary skew Young diagram. Let us show that 

Z\/v\n(x;y I a) = det[h Xi - H+j -i\ n (x;y | r^" J+1 a)]i< iJ <^ (A) . 

We use an appropriate modification of the Gessel-Viennot method (see [GV], [Sa, 
§4.5] for a detailed description of this method). 

Suppose we have 2n variables xi, . . . , x n , y±, . . . , y n . We define a path as a finite 
sequence p = (p(0),p(l), . . . ,p(l)) of points in the (x, y)-plane, such that: 

• Each point p(t) belongs to Z x {0, 1, 2, ... , n}. 

• The path starts at the horizontal line y = and ends on the line y = n, i.e., 

p(0) = (mi, 0) and p(l) = (777-2, n) with some mi, 7772 G Z. 
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• Each step p(t + l) —p(t) is of the form (1, 0), (1, 1) or (0, 1) (in accordance with 
these three options, we speak of a horizontal, diagonal or vertical step, respectively). 

• The first step p(l) — p(0) is either diagonal or vertical but not horizontal. 
We will deal with collections of paths L = (pi)i<j<£(A) such that pi starts at 

(/ij — i, 0), ends at (Aj — i, n), and PiHpj = if % ^ j . The set of all such collections 
will be denoted by L(X//j). 

With each collection L = (pi)i<i<£(\) G L(\/ fx) we associate a tableau T of 
shape X/ fx and of order n by the following rules: 

{pi{t) = (m- l,fc), Pi(t+ 1) = (m,fc)} =>• T(i,m + i) = k; 
{pi(t) = (m- l,k- 1), pi(t + 1) = (m, fc)} T(i,m + i) = k'. 

That is, the ith path codes the filling of the ith row in A///, each square 
being associated with a nonvertical step in pi. If the endpoint of a nonvertical step 
has coordinates (m,k) then j = i + m, and we have T(i,j) = k or T(i,j) = k' 
according to whether the step is horizontal or diagonal. 

To verify that T is indeed a tableau of shape \j \x and of order n we check the 
four properties of Definition A. 3. 

Properties (1) and (3) follow from the definition of the path pi. 

Let us check properties (2) and (4). Given two boxes and (i + 1, j) in A//i, 
we have to prove that if T(i,j) is primed then T(i + > T(i,j), and if T(i,j) is 
nonprimed then T(i + > T(i,j). Set m = j — i, k = \T(i,j)\, r = \T(i + 
The boxes and (i + 1, j) correspond to nonvertical steps Pi{t) — > pi(t + 1) and 
Pi+i(s) — > Pi+i(s + 1) such that pi(t + 1) = (m, fe) and Pi+i(s + 1) = (m — l,r), 
respectively. Since the paths and do not intersect, the point Pi+i(s + 1) 
must be strictly above the point Pi(t). This condition is exactly what we need. 
Indeed, examine the two possible cases: 

First, assume that T(i,j) is primed, T(i,j) = k' . This means that the step 
Pi(t) Pi(t + 1) is diagonal, and we have Pi(t) = (m — l,k — 1). Then we get 
r > k — 1, i.e., r > k, as required. 

Second, assume that T(i,j) is nonprimed, T(i,j) = k. This means that the step 
Pi(t) — > Pi(t + 1) is horizontal, and we have Pi(t) = (m — 1, fc). Then we get r > k, 
as required. 

Conversely, if T satisfies the four conditions (1) — (4) then, reversing the above 
argument, we conclude that the paths are pairwise nonintersecting. Thus, the 
correspondence L — > T is a bijection between L(\//j,,n) and Tab(A//i, n). 

Next, we assign to an arbitrary path p its weight H {p) as follows: the weight of 
p is the product of the weights of its steps, where 

• a horizontal step with endpoint (m, k) has weight Xk — a m ; 

• a diagonal step with endpoint (m, fc) has weight + a m ; 

• any vertical step has weight 1. 

For an arbitrary collection L = (pi)i<i<e(\) G L(A//i) we set 

<(A) 

n L = J]n(p,). 

i=l 

Then we get 

nL = Yl ( x T(n) - a c (n)) G/T(n) + a c (n))> 

T(n) = l,2,...,n T(n)=l',2',...,n' 
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where T <-> L. It follows that 



E A/M , n = ^ n L . (A.4) 

The standard argument of the Gessel-Viennot theory (see [GV], [Sa, §4.5]) shows 
that 

^2 H L = det[E(i,j)] 1 < i)i <*( A ), (A.5) 

L6l(A/(i,n) 

where E(z, j) = n(p), summed over all paths p starting at (fij —j, 0) and ending 
at (Xi —i,n). From (A. 3) it follows that 

= h Xi _^ +j _ i{n (x;y\ r^ +l a). (A.6) 

From (A.4), (A.5) and (A.6) we obtain that 

^\/v\n(x;y I a) = det[h Xt _ H+j _ iln (x;y | r^" J+1 a)] 1 < iJ <^ (A) 

and, consequently, 

s a/m (xi, x 2 , • • • , x n ; yi, y 2 , . . . , y n \ a) = E\/p\ n [x; y\a). 

□ 

References 

[BR] A. Berele and A. Regev, Hook Young diagrams with applications to combinatorics and to 
representations of Lie superalgebras, Adv. Math. 64 (1987), 118-175. 

[BL1] L. C. Biedenharn and J. D. Louck, A new class of symmetric polynomials defined in terms 
of tableaux, Advances in Appl. Math. 10 (1989), 396-438. 

[BL2] , Inhomogeneous basis set of symmetric polynomials defined by tableaux, Proc. Nat. 

Acad. Sci. U.S.A. 87 (1990), 1441-1445. 

[BO] A. Borodin and G. Olshanski, Harmonic functions on multiplicative graphs and interpo- 



lation polynomials, Electronic J. Combinatorics 7 (2000), #R28; nath/9912124. 
[CL] W. Y. C. Chen and J. D. Louck, The factorial Schur function, J. Math. Phys. 34 (1993), 
4144-4160. 

[GV] I. Gessel and G. Viennot, Binomial determinants, paths, and hook length formula, Ad- 
vances in Math. (1985), 300-321. 

[GG] I. Goulden and C. Greene, A new tableau representation for super symmetric Schur func- 
tions, J. Algebra 170 (1994), 687-703. 

[II] V. N. Ivanov, Dimension of skew shifted Young diagrams and projective representations 
of the infinite symmetric group, Representation theory, dynamical systems, combinatorial 
and algorithmical methods II (A. M. Vershik, ed.), Zapiski Nauchnykh Seminarov POMI 
240, Nauka, St. Petersburg, 1997, pp. 115-135 (Russian); English translation: J. Math. 
Sci. 96 (1999), no. 5, 3517-3530 

[12] V. N. Ivanov, Combinatorial formula for factorial Schur Q-functions, in: Representation 
theory, dynamical systems, combinatorial and algorithmical methods III (A. M. Vershik, 
ed.), Zapiski Nauchnykh Seminarov POMI 256, Nauka, St. Petersburg, 1999, pp. 73-94 
(Russian); English translation: J. Math. Sci., to appear. 

[KO] S. Kerov and G. Olshanski, Polynomial functions on the set of Young diagrams, Comptes 
Rendus Acad. Sci. Paris Ser. I 319 (1994), 121-126. 

[KOO] S. Kerov, A. Okounkov, G. Olshanski, The boundary of Young graph with Jack edge 
multiplicities, Intern. Math. Res. Notices (1998), no. 4, 173-199. 

44 



[KV1] S. Kerov, A. Vershik, The characters of the infinite symmetric group and probability prop- 
erties of the Robins on- Schensted-Knuth algorithm, SIAM J. Alg. Discr. Meth. 7 (1986), 
116-124. 

[KV2] , The Grothendieck group of the infinite symmetric group and symmetric functions 

with the elements of the Ko-functor theory of AF-algebras, Representation of Lie groups 
and related topics (A. M. Vershik and D. P. Zhelobenko, eds.), Adv. Stud. Contemp. Math. 
7, Gordon and Breach, 1990, pp. 36-114. 

[LI] A. Lascoux, Puissances exterieurs, determinants et cycles de Schubert, Bull. Soc. Math. 
France 102 (1974), 161-179. 

[L2] , Notes on interpolation in one and several variables, Preprint, available via 



tittp : / / schubert . univ-mlv . f r/ ~al/MAIN/publications . html 



[Mai] I. G. Macdonald, Symmetric functions and Hall polynomials, 2nd edition, Oxford Univer- 
sity Press, 1995. 

[Ma2] , Schur functions: theme and variations, Publ. I.R.M.A. Strasbourg , 498/S-27, 

Actes 28-e Seminaire Lotharingien (1992), 5—39. 
[Ma3] , Notes on Schubert polynomials, Publ. LACIM, Universite du Quebec, Montreal, 

(1991). 

[Mo] A. Molev, Factorial super symmetric Schur functions and super Capelli identities, Ki- 
rillov's Seminar on Representation Theory (G. Olshanski, ed.), American Mathematical 
Society Translations (2), Vol. 181, Amer. Math. Soc, Providence, R.I., 1997, pp. 109-137. 

[MS] A. I. Molev and B. E. Sagan, A Littlewood-Richardson rule for factorial Schur functions, 
Trans. Amer. Math. Soc. 351 (1999), 4429-4443. 

[Okl] A. Okounkov, Quantum immanants and higher Capelli identities, Transformation Groups 
1 (1996), 99-126. 

[Ok2] , On Newton interpolation of symmetric functions: a characterization of interpo- 
lation Macdonald polynomials, Adv. Appl. Math. 20 (1998), 395-428. 

[OOl] A. Okounkov and G. Olshanski, Shifted Schur functions, Algebra i Analiz 9 (1997), no. 2, 
73-146 (Russian); English translation: St. Petersburg Math. J. 9 (1998), 239-300. 

[002] , Shifted Jack polynomials, binomial formula, and applications, Math. Research 

Letters 4 (1997), 69-78. 

[Ol] G. Olshanski, Point processes and the infinite symmetric group. Part I: The general for- 
malism and the density function, nath/9804086. 
[ORV] G. Olshanski, A. Regev, and A. Vershik, Frobenius-Schur functions: summary of results, 



math/0003031 



[P] P. Pragacz, Alg ebro- geometric applications of Schur S- and Q -polynomials, Topics in 
Invariant Theory, Seminaire d'Algebre Paul Dubriel et Marie-Paule Malliavin, Lecture 
Notes in Math., vol. 1478, Springer-Verlag, New York/Berlin, 1991, pp. 130-191. 

[PT] P. Pragacz and A. Thorup, On a Jacobi-Trudi identity for supersymmetric polynomials, 
Adv. Math. (1992), 8-17. 

[RS] A. Regev and T. Seeman, Shuffle invariance of the supei — RSK algorithm, Advances in 
Appl. Math., to appear, fnath/010320e . 

[Sa] B. E. Sagan, The symmetric group. Representations, combinatorial algorithms, and sym- 
metric functions, Brooks/Cole Publ. Co., Pacific Grove, CA., 1991. 

[St] J. R. Stembridge, A characterization of supersymmetric polynomials, J. Algebra 95 (1985), 
439-444. 

[T] E. Thoma, Die unzerlegbaren, positive-definiten Klassenfunktionen der abzdhlbar un- 
endlichen, symmetrischen Gruppe, Math. Zeitschr. 85 (1964), 40-61. 

[VK] A. M. Vershik and S. V. Kerov, Asymptotic theory of characters of the symmetric group, 
Funct. Anal. Appl. 15 (1981), no. 4, 246-255. 

[W] A. J. Wassermann, Automorphic actions of compact groups on operator algebras, Thesis, 
University of Pennsylvania (1981). 



G. Olshanski: Dobrushin Mathematics Laboratory, Institute for Information 
Transmission Problems, Bolshoy Karetny 19, Moscow 101447, GSP-4, Russia. 
E-mail address: olsh@iitp.ru, olsh@online.ru 

A. Regev: Department of Theoretical Mathematics, Weizmann Institute of 
Science, Rehovot 76100, Israel. 

45 



E-mail address: regev@wisdom.weizmann.ac.il 

A. Vershik: Steklov Mathematical Institute (POMI), Fontanka 27, St. Peters- 
burg 191011, Russia. 

E-mail address: vershikOpdmi . ras . ru 
V. Ivanov: Moscow State University. 
E-mail address: vivanov@vivanov.mccme.ru 



46 



